IRC channel logs
2023-03-08.log
back to list of logs
<stikonas>so my dmesg shows 1 segfault in libtool, one in conftest and loads in gnulib-tool <stikonas>pder: so this bit is crashing bash in gnulib-import: <stikonas>so we just have associative arrays disabled <pder>env -i ./bash also segfaults <stikonas>ok, I've got a backtrace for that declare -A line <stikonas>pder: do you have something in your .bashrc? <stikonas>that is sourced automatically and maybe uses associative arrays <pder>interesting, so its something in my .bashrc because bash --norc works <stikonas>anyway, my guess is that this is the same cause <stikonas>and you can probably just debug that reproducer: declare -A x && x[f/2]='foo' <oriansj>alethkit: I finally had a chance to give a proper look at the guile steel proposal; and if one wanted a Systems Lisp that is already done, SBCL is an excellent choice. If your intent is something different, I'd need more to determine if I know of anything that is readily available to suit that need. <pder>After disabling bash_completion by commenting out . /etc/bash_completion in my .bashrc there is no segfault <rickmasters>stikonas: on Fiwix the automake 1.15.1 configure script hangs. It was segfaulting in a different place before I added some diagnostics. <rickmasters>I'll be honest. I've been working on this bash 5 upgrade for a few days and I'm running out of motivation. <fossy>rickmasters: i'm not really super in the loop of this issue, is it fiwix + automake 1.15 configure script + bash 5 = crashes? <fossy>that is. odd to say the least <rickmasters>Its kind of optimistic that bash 5 would just drop onto Fiwix and work without significant effort being invested. <fossy>yeah, but why automake 1.15 configure script specifically.. <rickmasters>Its the first significant bash script that runs after bash is built. <fossy>oh, it is the first configure script in run2.sh <rickmasters>bash is complicated and uses a lot of kernel facilities. I'm not sure I'm up for making it work right now. Have to be honest. <fossy>rickmasters, stikonas: new bash is a nice to have in my opinion, rather than a nessecity. i think probably push new bash back to sysc for the time being, and come back to look at what versions will work on fiwix later... <fossy>yeah, and it's relatively minor; i don't want this to be a blocker for your pr <rickmasters>well, I it was part of a dependency chain that gets us binutils 2.30 I think. I hate to hold up progress <rickmasters>I just don't know how much time or effort I'd be on the hook for here. <pder>I think the software you build in Fiwix should stay as minimal as possible with the goal of building the linux kernel <fossy>minimal and old are two different things <fossy>binutils 2.30 replaced binutils 2.14 as the first version of binutils <rickmasters>Yeah, putting bash 5 before kernel bootstrap is a tough pill. I'd be willing to keep working on it as a side project but I think reverting it for now would get me back on track with the kernel bootstrap work <pder>even if it means building an older linux kernel, building newer binutils, bash, etc, and then a newer linux kernel seems easier to me <fossy>rickmasters: i'll get back to you shortly and see if we can revert just parts of the binutils 2.30 upgrade -- feel free to revert it as a whole locally if it'll help you get back on track <fossy>pder: well, we need *a* version of binutils, to build any linux kernel, but it doesn't need to be 2.30 <fossy>in saying that, 2.30 is highly desirable to have eventually, since it has riscv support <pder>what is the reason binutils 2.30 needs bash 5? Was there something in bash 2 that was needed to run configure scripts? <fossy>the dependency problem if it exists would be <fossy>bash 5 -> automake 1.15 -> binutils 2.30 <pder>For now it makes the most sense to me to keep sysa stable and not add complexity until the kernel stuff is merged, and then revisit this <fossy>this moves bash 5 back to sysc and fixes bash 5 -> automake 1.15 dep <fossy>the configure script is lying when it says bash 5 is required <rickmasters>fossy: I can try that but I'll have to redo some merges and I probably won't know until tomorrow <fossy>rickmasters: no worries, hopefully this unblocks kernel stuff <rickmasters>Mikaku: is that stock bash 4.4.18 you have running or is modified for Fiwix in some way? <rickmasters>So my latest test did something funny. configure was hanging on a very large Heredoc starting with cat >>$CONFIG_STATUS <<\_ASEOF <rickmasters>So i broke it up into different sections to see what part was causing the problem <rickmasters>But that caused it not hang. There was still a page fault but automake-1.15.1 did finish. <rickmasters>But then the configure script for binutils-2.30 did the same thing - its frozen at the same place. <rickmasters>Anyway, its still not working but I thought that was interesting. <rickmasters>Oh, just to be clear this is the bash 5 issues not the branch you made fossy <stikonas[m]>Yes, we can later update to bash 4 but let's merge kernel work first <Mikaku>rickmaster: I've also experienced some hangs or segfaults when running (big sized) ./configure in some packages <Mikaku>rickmasters: it looks to me that it must be something related to the stack but sincerely I've been unable to find it out <Mikaku>rickmasters: I gave it up with the hope that if it's really related to the stack, it would appear in other places and hopefully easier to catch <alethkit>oriansj: I think the point is that SBCL requires a C compiler, whereas this does not. <msavoritias>That sounds like a good point. Then it can make bootstrapping easier. In theory at least. <stikonas>but is there actual guile steel or just the proposal? <stikonas>msavoritias: in practice a lot of lisps tend to have pregenerated files that need preexisting lisp and they often don't bother solving it <stikonas>yes, but bootstrapping this thing without C might not be easy either <stikonas>can one write an assembly code in less than 5k LOC (including comments and labels) that can run this... <msavoritias>Good question. Also apparently this just translates it to c so not sure how much that helps <stikonas>it might help for other things (performance maybe) <stikonas>for bootsrapping purposes you should thing that you have some simple assembly language and you want to write some higher level language in it <stikonas>(well, trivial shell or kernel if you do posix/some other os bootstrap) <msavoritias>Cant we put a simple low level in between? Like carp or scopes? <msavoritias>Because i am thinking making a gc into assembly complicates things <stikonas>well, you could have something between assembly and higher lisp <msavoritias>Which sounds like pre-scheme but mapping to assembly instead of c <stikonas>still that first step must be able at least a few things <stikonas>at the very least we need to be able to open files, read and write to them <stikonas>and if it's compiled language, then it has to support inline assembly to do those i/o syscalls <msavoritias>Still if prescheme is able to bootstrap guile that could simplify things <msavoritias>Especially if guile can be "ported" to use prescheme to bootstrap itself <stikonas>I thought it has quite a few C dependencies too <stikonas>anyway, quite a lot can be done in bootstrapping if somebody is willing to do those things. But there are often some details that people initially overlook. For example tcc compiler can build gcc. But in live-bootstrap we have another 50 steps between tcc and gcc which people might initially overlook <stikonas>it's stuff like perl and autotools, bootstrapping bison/flex, C libraries, binutils <msavoritias>Agreed. I am talking on high level from here. And as you said guile itself has quite a bit of c code and of c deps <msavoritias>Plus if guix and everything else on userspace requires c anyway you are starting to question the benefits <stikonas>yes, majority of userspace does need C... <stikonas>which means you'll need to write a C compiler capable of building GCC <msavoritias>My goal personally would be to run up until guix and shepherd without messinp with c compilers <msavoritias>Or if i want to build guix in a container not to have to built c from scratch <oriansj>alethkit: well the only lisps written in assembly either are for architectures that are "dead" or too simple to be used in the traditional lisp way. (I know I've written in assembly one too [it is in stage0 in a file named lisp.S]) <oriansj>so msavoritias you will need to write a protolisp in assembly and I have an s-expression lisp spec you could implement in assembly in 24 hours if you are experienced in assembly and sufficiently motivated to do the work. <AwesomeAdam54321>oriansj: What's missing from your lisp.S before it could be used as part of a bootstrap? Is it I/O? <msavoritias>But i am reading and learning to do it at some point. <msavoritias>But why not have pre scheme just compile to assembly instead <AwesomeAdam54321>msavoritias: There isn't a problem with that, it's just that pre scheme would also have to be implemented in assembly(or simpler language) too in that case <msavoritias>More like implement instead of pre scheme compile to c to compile to assembly but yeah <oriansj>AwesomeAdam54321: it can be used as part of a bootstrap as is. But it is a lisp interpreter not a compiler and thus one would need to write a lisp compiler in its subset. <oriansj>msavoritias: compiling to C is trivial; as C handles all of the messy details. Compiling to Assembly takes a good deal more as you can clearly see in every compiler written in Lisp. <stikonas>well, that's because it's basically a portable assembly <stikonas>so I would say compiling to assembly is worse than compiling to C <stikonas>still, you could do lisp interpreter for some time <stikonas>but then your interpreter itself would be written in assembly <stikonas>and any sycalls will eventually have to go through original interpreter <msavoritias>I disagree. C was built for computers 60 years ago that no longer exist. Webassembly sounds closer to what a portable assembly would actually look like. <stikonas>though perhaps it is suitable for that output format for prescheme <stikonas>but then you need to convert webassembly to machine code too <stikonas>ok, there is also web assembly text version... <stikonas>I haven't looked at webassembly before, but maybe M0 can be useful in converting text webassembly to hex version <stikonas>perhaps implementing webassembly in assembly is easier than writing cc_* compiler <stikonas>but then again, somebody has to redo the work <stikonas>and if somebody does it, I'm sure people here would consider it and won't reject <msavoritias>Writing a webassembly compiler in assembly if it doesnt exist. And then prescheme can output to that <msavoritias>And then if we have that webassembly compiler we can put whatever input we want. Lots of reading to do for this though :D <stikonas>still that's assuming that we can write webassembly compiler that is smaller than cc_x86 <stikonas>(or cc_amd64 or any other implementation of cc_*) <oriansj>well decoding webassembly binaries would be more complex than cc_* but converting webassembly text to native binaries might be simple enough <oriansj>but you might need to limit yourself to a subset <oriansj>the i64.load* instructions would definitely complicate implementations on 32bit architectures <stikonas>it doesn't have to be exactly webassembly, just something similar of your choice <stikonas>though if it's too custom, it might be harder for others to understand <oriansj>well webassembly is effectively just an array FORTH <stikonas>and one would have to write a webassembly text "C" library <stikonas>(that's probably not part of webassembly but we could add an extension) <oriansj>or write a webassembly output for M2-planet to figure out the mapping <oriansj>but the step converting theory to reality, usually eliminates false assumptions. <rickmasters>fossy: the bash-automake-junk branch works better on Fiwix. But there is something wrong with that branch not related to Fiwix. <rickmasters>fossy: it fails on bc-1.07.1 apparently running ./fix-libmath.h with: make[2]: ./x86/bin/kaem: Shell program not found <rickmasters>fossy: this happens on Linux chroot and qemu (standard, non Fiwix) <stikonas>hmm, we saw this error in this channel a month or so ago... <stikonas>rickmasters: perhaps fossy hasn't finished that branch, there is no PR <rickmasters>fossy: With Fiwix, I do get a page fault from make in the automake-1.15.1 build but I also see an error on Linux: make: find: Command not found <rickmasters>But presumably its working on the main branch somehow? <stikonas>I think find is only used to set permissions in automake 1.15 <rickmasters>stikonas: maybe the main branch has a fixed path due to your bash and .env changes? <stikonas>building diffutils really needs binutils <stikonas>so we have autotools before binutils before findutils <rickmasters>stikonas: or maybe bash 5 exports a SHELL variable or something that makes no-shebang work? <stikonas>rickmasters: also sam_ suggested switching to another bc implementation <rickmasters>fossy, stikonas: I verified that SHELL is ./x86/bin/kaem on bash-automake-junk and is /bin/sh on main branch <stikonas>(though maybe we should also consider using bash 4 that is known to work on fiwix) <rickmasters>I'll put in a workaround for my Fiwix testing. I just want to make sure Fiwix can still build Linux with fossy's approach. <stikonas>well export SHELL=/bin/sh should do it... <rickmasters>stikonas: By the way, I noticed that compiling bash without job control reveals a bug that orphans processes in pipelines <rickmasters>stikonas: I was running into process limits on Fiwix and it was because bash was only waiting on the last command in a pipeline to finish, the others were orphaned. <rickmasters>Normal init cleans up orphans but ours is kaem which does not, so they would build up. <rickmasters>bash compiled with job control waits on all processes in a pipeline <rickmasters>Not that I'm asking for job control - it may use library or system calls we don't have but I am wondering if you know why we compiled bash that way. <stikonas>that's mostly because it's handwirten makefile <stikonas>so we need to figure out which preprocessor defines need enabling <rickmasters>stikonas: I think I tried briefly to compile it with job control but ran into troubles I don't remember so I just increased Fiwix process limits. <stikonas>but yes, this sounds something to look at... <rickmasters>fossy: the bash-automake-junk branch works on Fiwix through Linux but it needs "export SHELL=/bin/sh" near the top of sysa/run.sh <rickmasters>fossy: also that branch needs a binutils-2.30 checksum <stikonas>at least binutils 2.30 is not causing any problems <stikonas>bash we'll be able to sort one way or another... <stikonas[m]>But at least we do have a better understanding of fiwix limits <rickmasters>stikonas: no worries. I'll still keep poking at it when I have time. Hopefully it can be made to work some day. <stikonas[m]>Bash 4 might be better for new features / time spent ratio <stikonas[m]>If I remember bash 5 doesn't have too many new things anyway