IRC channel logs

<oriansj>Store_String only allocates 256bytes (buffer overflow)

<oriansj>the line is 293 chars in length

<oriansj>up to 512 -> boom problem sovled

<stikonas>ok, and I found the issue in cc_*

<stikonas>it fixes that string to be normal

<stikonas>oriansj: https://github.com/oriansj/stage0-posix/pull/47

<stikonas>are you incraesing the buffer? or should I do that?

<oriansj>I'm thinking of what the correct thing to do is. the simplest would be unify the max string size to match the M1 default (4K) but because M0 doesn't deduplicate nor use a scratch it ends up being a good bit bigger in terms of RAM (about 300MB if we do 4K but only 16MB if we do 256)

<stikonas>well, 4K is probably too much

<stikonas>especially since M0 does not compile large programs...

<stikonas>M1 might be used to build something much bigger

<stikonas>maybe 512 would be a good compromise?

<oriansj>and 16MB of RAM is far easier to bootstrap in regards to core memory or current libre lithography process than 300MB

<stikonas>that's what AArch version is using

<stikonas>hmm

<stikonas>true...

<oriansj>and having different values would enable us to notice the differences between architectures to highlight if we change M2-Planet in a way that breaks the 256 boundary like this in the future.

<stikonas>so we should keep it at 256?

<stikonas>well, in principle that should be enough if cc_riscv64 is implemented without bugs

<oriansj>unless I modify M2-Planet's test1000 to also check for too large strings in M1.

<stikonas>although, let me test if M0 now works fine on M2.M1

<stikonas>hmm, actually still crashes

<oriansj>the question is does M1 and M0 output (when built by the full C hex2 produce the exact same output)

<stikonas>no :(

<stikonas>let's make a diff...

<stikonas>oh, different line breaks...

<stikonas>let me compare final binaries...

<oriansj>(hence why I suggested after both pass through the full C hex2)

<oriansj>as whitespace differences are expected between M0 and M1 but the hex2 inputs should functionally be identical

<stikonas>yes, final binaries are identical

<stikonas>but that's when M0 does not crash...

<stikonas>let me see where it still crashes

<stikonas>yeah, still crashes in Reverse_List

<stikonas>and still garbage in brk pointer...

<stikonas>so I think you found a different bug with those weird strings

<stikonas>not even 512 byte buffer helps

<oriansj>well one stupid hack is have brk add 512 to all non-zero sized requests

<oriansj>and if the garbage still exists it is because the kernel isn't zeroing memory before passing it to our process.

<oriansj>which will eliminate all under allocation problems (or add 1MB if you are really worried)

<stikonas>no, adding 512 does not help here...

<stikonas>maybe it's qemu translation layer messing up something

<stikonas>hmm

<stikonas>not too sure about what qemu does exactly

<stikonas>although, it might be crashing less frequently

<stikonas>only my first run crashed

<oriansj>make all allocations a multiple of 8?

<stikonas>ok and another

<stikonas>hmm

<oriansj>add 512 then and 0xFFFFFFF8

<stikonas>let me first reduce string buffer back to 256

<stikonas>nope, that doesn't help either

<stikonas>crash in #0 Reverse_List_Loop () at M0_riscv64.S:257

<stikonas>but that's only because it's the first place where dereference happens

<oriansj>dereference?

<oriansj>next is a direct pointer

<stikonas>well, a1 already contains some garbage at that time

<stikonas>I have $a1 = 0x646e655f464c453a

<oriansj>does the load not set the whole register?

<stikonas>it should

<stikonas>ld is 64 bit

<stikonas>(lw is 32 bit) and lb is 8 bit

<stikonas>and M0 is able to build cc_riscv64 just fine... So must be quite corner-case bug

<stikonas>I still suspect something with brk

<stikonas>as that malloc is the only function that writes to s4 register

<stikonas>and s4 contains garbage

<stikonas>sorry, I meant s1

<oriansj>syscalls might set s4

<stikonas>but s1 and s4 have same stuff

<stikonas>no, I don't think syscalls touch s registers

<stikonas>they return stuff in a registers

<stikonas>normally a0

<oriansj>what registers does linux syscalls actually use

<stikonas>a7 is syscall number then a0, a1, a2, ... a6 are syscall arguments

<oriansj>because any register used in a syscall maybe modified during any syscall

<stikonas>well, it returns a0 and then we do mv s1, a0 # Set our malloc pointer

<oriansj>so any syscall can alter a1 and a4

<stikonas>yes, that's true

<stikonas>a4 is actually completely unused

<stikonas>for some reason I have it only once pushed and popped from the stack

<stikonas>but not used at all, so those lines can be removed

<oriansj>well the pops at the end of Reverse_List are to undo all register changes except the register used to return the reversed list

<oriansj>one thing you can do is have gdb break when S4 changes

<oriansj>assuming it shouldn't change much

<oriansj>correction S1 changes

<oriansj>watch S1 should do it

<stikonas>well, it does change quite a bit

<stikonas>once per token

<stikonas>there are 234233 words in M2.M1

<stikonas>well, some are comments but there are still a lot...

<oriansj>the top 32bits from malloc should always be zero right?

<stikonas>hmm, I guess so

<oriansj>so only break if the top 32bits are anything other than zero

<stikonas>but I don't understand yet why that happens...

<oriansj>well breaking when it does, lets us know where to give us more clues

<stikonas>oh I see you mean stop in debugger...

<stikonas>yeah, I can try that....

<stikonas>although, maybe tomorrow

<pabs3>could someone apply this patch to the website? https://paste.debian.net/plain/1213263/

<janneke>pabs3: thanks. i've completed the commit messages by adding the changelog and and applied the patch set

<pabs3>janneke: the commit messages were already in the patchset, there were two commits

<pabs3>ah, I see what you mean, you copied the list of changed files into the commit message. I generally just use `git log --stat` when I want to see that.

<pabs3>thanks for merging

<janneke>yw, yes we appreciate and for some projects require gnu style changelogs, see https://guix.gnu.org/manual/en/html_node/Submitting-Patches.html and https://www.gnu.org/prep/standards/html_node/Change-Logs.html#Change-Logs

<pabs3>I see

<NieDzejkob>hot take: gnu-style changelogs are a relic from the era of VCSes not existing and the only reason they're still being used is that nobody realized that they don't make sense anymore

<oriansj>NieDzejkob: there is a big difference between not making sense anymore and not being in a form that optimally matches the tools that are available *now*

<oriansj>learning how to shoe a horse is a skill that in a public school would fall into the not making sense anymore category. Whereas teaching of visual basic would be in a form that doesn't optimally match the tools that are available now.

<oriansj>I'd argue gnu-style changelogs are more of not being optimal than being senseless these days; as there is great value in good commit messages and commit patterns in a style that makes it easier for others to disect and understand. A skill I hope to one day properly learn (if abit slowly at this point)

<NieDzejkob>Listing the functions and files the commit touched doesn't "make it easier for others to understand"

<oriansj>NieDzejkob: but "What was the rationale for a given change, and what were its main ideas?" certainly does

<NieDzejkob>yes, but that's the opposite of the GNU changelog format.

<oriansj>but it is literally what it says: https://www.gnu.org/prep/standards/standards.html

<oriansj>another good bit: Is there any additional information regarding the change, and if so, where can it be found?

<oriansj>and suggests: one commit for each logical changeset

<NieDzejkob>hmm. Apparently what the document actually says and what the Guix folks are doing while citing the document are two entirely different things.

<oriansj>yes it is called using a previously learned skill without updating it to match the new improved standards.

<civodul>NieDzejkob: it's not quite that they don't make sense anymore; they give a view of the intended language-level changes, as oppose to line-by-line changes

<oriansj>as scheme isn't so much a programming language but as glue for a community to create its own langague to express the solving of a problem.

<oriansj>stikonas: fsopen is RISC-V's open? #430 or is my partially construction documentation completely off: https://paste.debian.net/1213370/

<stikonas>oriansj: I've used openat

<stikonas>fsopen seems to be something to do with file systems

<oriansj>well that is what an open command does. it opens a file in a file system.

<stikonas>and fsopen seems to open a file system, not a file in filesystem

<stikonas>i.e. for mounting purposes

<stikonas> https://lwn.net/Articles/759499/

<oriansj>thanks, learned something new

<stikonas>well, I didn't know this either

<stikonas>and I just spend lots of time resetting up my openwrt router after upgrade refused to carry over old configuration

<stikonas>so didn't really have time to look at M0 crashe yet...

<stikonas>oriansj: but this is probably worth pulling in https://github.com/oriansj/stage0-posix/pull/47

<stikonas>otherwise we will be hitting too long string bug

IRC channel logs

2021-09-26.log