IRC channel logs
2026-03-15.log
back to list of logs
<mwette>nyacc 3.04.5 released; I pray this is the working one for you <matrix_bridge><wildwestrom> Andrius Štikonas: Alright then. The way I've been doing it was writing GNU assembler source, then converting that to hex and cross-referencing objdump from GNU binutils. <matrix_bridge><wildwestrom> Alright, I used an automated python tool to check each and every byte so that I can understand what each line does. <matrix_bridge>Then I added many more comments to hex0. At this point I think I can consider it audited. <snuik>Welcome back janneke you have 2 messages <snuik>janneke, ekaitz says: arbitrary length or many-buffer? I think we need the latter rather than the former <snuik>janneke, ekaitz says: I just read the patch: so it does both. Couldn't we just have it statically allocated? if it is a global I don't see why we shouldn't... I don't understand that `malloc` <janneke>ekaitz: i'd like a review of that especially as there are several perspectives <ekaitz>janneke: how does glibc do it? and musl? <janneke>the posix c standard requires only buffer of length 1 <janneke>mwette is using unget-char from scheme...and scheme supports an unlimited buffer... <ekaitz>janneke: maybe then we should implement that in scheme? <janneke>however, as mes lib c and mes are one project, and reads, buffered read,s and unget is already a pretty complex problem, it seems to me that it's "handy" to add any unget feature in one place only <janneke>ekaitz: we do that for any port other than file ports, or stdin, dunno exactly (sorry!) <janneke>it could be a bug or thinko, but mes doesn't support unlimited unget for stdin *or something* <janneke>so, stefan emailed me several proposals for a much smaller 4 byte ungetc buffer...but i believe we need at least shorts because unget could also be EOF, which is beyond a char <janneke>i've now set UNGETC_MAX to 10, where NYACC only ever needs 2, i believe... <janneke>but we could set it to 1000 and so claim "mostly scheme compliance" iow, wdyt? <janneke>so many lovely options that all kinda suck :) <ekaitz>janneke: i think the cleanest solution is to make a scheme unget separatedly <ekaitz>which actually supports everything it should <ekaitz>now, maybe being clean is not the best solution <janneke>if we can figure that out, we can revert my HACK patch for mes lib c <ekaitz>yes, for the moment your patch looks ok <janneke>yes, so there's this pragmatic approach: release 0.27.2 including this HACK patch, and add to ROADMAP to revert this patch and do unlimited unread in scheme <ekaitz>also, is the unlimited unread mandatory in scheme? (is there anything mandatory in scheme?) <janneke>i think i like it too, so thanks for bringing this up, i hadn't figured out this option <janneke>Scheme Procedure: unget-char port char <janneke> Place character char in port so that it will be read by the next read operation. If called multiple times, the unread characters will be read again in last-in first-out order. " <janneke>ekaitz: and then there's unget-string, with a similar description <janneke>IWBN for the guile documentation to annotate with every procedure with which R*RS it complies or something <janneke>most of the time i'm lazy, see what guile does, and then "just do that" at least to the extent that NYACC needs it <janneke>"lazy" because i already work 70+h a week :) <janneke>hopefully except for those who are young parents <3, especially they should work less! <janneke>iow, ideally let everyone contribute to the extent of their ability <ekaitz>janneke: why do you work that much? <janneke>ekaitz: because i try to do only thing that i love to do and believe are important <ekaitz>yeah... but you should also have time for yourself, cooking, sleeping, etc <janneke>sure! since i've seen the light and gone vegan three years ago, i spend even more time cooking, i'm getting somewhat better at allowing myself to sleep as much as i need, going to bed at even at 21:00 when i'm tired...but i also find myself waking up at 4:00 and going to work :) <janneke>also, i try to walk for 1.5h a day; that works better in summer than in winter <janneke>ekaitz: some 205 ago, in my lilypond-time, i used to work from 19:00 until 4:00 three or four days a week, getting up at 7:00 going to work for my 40h job <janneke>sleeping only three hours a day, three or four days a week, i cannot do that anymore <ekaitz>i'm glad that I don't have a 40h job <ekaitz>i need a lot of time for myself, i have many things that interest me <janneke>i've tried to adopt the uberman sleep schedule two times, but just did't have enough time to make the change, due to external factors <janneke>iow, that schedule of working ~100h a week, my body can no longer sustain :) <ekaitz>i also work many hours or spend many hours a day reading, studying and all, but I don't code that muc <ekaitz>i just think or go for a walk or so <janneke>i use coding in two ways: hacking, trying many things, getting terrible results, just to gain understanding <ekaitz>and I don't think if I focused on working like 40h/week i would be more effective <ekaitz>oh yes deleting code is frustrating <janneke>after that, there can be a phase that you have enough understanding to "have the lymbdic system completely solve the problem", that can take days or weeks <janneke>you "see" the solution, and just code it -- done <ekaitz>i realized you need a low-volume of work in order to be able to do that <ekaitz>if you have pressure, things don't work as well <janneke>yes, or "mindless work", or free time <janneke>it's like povererty, it drives you to make terrible decisions <janneke>i believe i've told you this before, a good friend of mine, tiggr, had a company called "programmers without deadlines" <janneke>i find it funny, and sad, and hopeful (because we will beat them!), and amazing, and many other adjectives that "companies" don't seem to "get" this <janneke>add stress, deadlines, revoke responsibility, and even the most brilliant people will produce mediocre results at best <ekaitz>most of the times when I work for a company as a freelance they are suprised about how efficient I am and the amount of work I can do <ekaitz>when I worked in a company I was most of the time just sitting there, doing nothing <ekaitz>they would be surprised how "bad" I am in their terms now if they knew what my schedules are and how I work <ekaitz>it's just absurd how people are literally hostage of their timetable <janneke>same for me now i've incooperated with rutger in reasonable-sourcery.coop <janneke>we spend an enormous amount of time "just talking" with eachother <janneke>which is fun, and in the end, amazingly efficient <janneke>the more responsibility you have, the more efficient and effective you get <janneke>why the fuck would you waste your own time? <ekaitz>that question is very interesting <ekaitz>why would you waste other people's? <janneke>yeah, i wouldn't, but i've worked in companies... <ekaitz>civodul: not a very good answer though <ekaitz>then I'd like to remove tcc altogether hehe <janneke>still, removing my 9yo fork with 70+ patches is kinda great an an intermediate step, right? <janneke>ekaitz: have you seen: "<janneke> and also, imho, we should, in our long term planning, have a way to remove tinycc from the full-source bootstrap as it's antagonistic to our cause and thus a liability " <janneke>(there are some comments preceeding this making this case) <ekaitz>oh they rejected stefan's patches <janneke>ACTION has really no freakin' idea what's driving them <janneke>you could be enlarging your small community with these bootstrapping fools, at a very small cost, but no <ekaitz>passive aggressive in the communication <ekaitz>which is not related with the previous, but is also a concern <janneke>it's not related, but the fact it's not getting addressed is also not unrelated, imo <janneke>maybe /me shouldn't shout too hard about code that's "hard to read" but at least i believe i very, very much encourage better/more readable code <Googulator>ekaitz: I beg to disagree on removing tcc in favor of mescc - for me, removing Scheme would be a far bigger priority <Googulator>Unless someone can magically write a performant Scheme interpreter that actually makes mescc & gash performant - but I'm not holding my breath for that one <janneke>Googulator: the whole idea of guix, and mes, is the dream of the lisp machine <gabif>we don't have to stick to only one bootstrap path i guess <janneke>initially, mes went from assembly straight to scheme, and that's the ideal <janneke>germ0 and germ, or a merge with those, could be a step towards that dream <Googulator>Lisp will never perform well on our current CPU architectures. <janneke>chez scheme is for certain tasks at a factor of 4-6, while guile+hoot+wasm promise to do a factor of two better <janneke>Googulator: people are too stupid to recognize that imperative programming is a very, very bad idea <janneke>and there's the irrational fear of parens <janneke>which are, if you understand them, truly an amazing godsend when refactoring <janneke>it's next to impossible to "convince" someone that they really want to have a lisp (scheme) machine until they'r actually a scheme convert <Googulator>Modern CPU architectures are all built with VAX-like programmability in mind, not Lisp machine-like. We do have optimizations for functional programming, but they are for Haskell, not Lisp. <Googulator>(too bad the Haskell community is hostile to bootstrapping...) <Googulator>Unless we can convince CPU manufacturers to optimize for Lisp, I don't see a performant Scheme as possible. <Googulator>Also, Haskell is able to be performant because of true native code generation thanks to Lennart Augustsson's work (the G-machine), AFAIK no such thing exists for Lisp. <stikonas>anyway, if we have more capable compilers, it's fine if we have multiple bootstrap paths <ekaitz>Googulator: mes will be faster, if that means something <Googulator>"Faster" isn't sufficient - we need "competitive with C". <Googulator>E.g. a Scheme implementation of gzip that's 5 times slower than the C one is acceptable for bootstrapping, but IMO one that's 1000 times slower isn't, even if that's already a 5-fold improvement over the current version of Mes + gash-utils <ekaitz>if that's the question the answer is no <ekaitz>i'm not sure if that's even *possible* <ekaitz>it's not a problem with scheme but with some decisions Mes does <ekaitz>what are those optimizations that haskell has that lisp cannot have? <Googulator>True machine code generation based on the G-Machine work <Googulator>Compiled Haskell programs only need a library at runtime, not a bytecode interpreter <ekaitz>you can compile lisp to x86_64 if you want <Googulator>Python can also be "compiled" in the loosest sens of the word, but I wouldn't really call that compilation <ekaitz>are you familiar with Chicken, Gambit... <Googulator>no... don't those just output a copy of the interpreter with byte code packed in the executable? <ekaitz>those compile to C, and that C program is compiled to x86_64 <ekaitz>but you could step the C step if you wanted <ekaitz>scheme can be VERY fast comparing to any other language with the same level of abstraction and flexibility <ekaitz>even more, chez, which is not compiled to machine code, it's probably faster than Haskell in some cases <Googulator>wait, I thought transpiling Lisp to C or LLVM IR is an unsolved problem in computer sciences... <Googulator>which is the whole reason Lisp machines were created in the first place <ekaitz>you weren't very informed in that case <ekaitz>i have a book right here that teaches you how to do it <ekaitz>wingo has compiled guile to wasm and wasm to C <Googulator>in that case, why did Lisp machines need a dedicated CPU architecture, rather than just an operating system on top of e.g. PDP-11? <ekaitz>i don't work at Genera, but it could be that, it could be marketing, it could be they had some extra stuff, it could be the PDP-11 was too slow for the loss during compilation... <stikonas>Googulator: I guess part of the slowdown comes from M2-Planet <stikonas>it emits basically to non-optimized code <janneke>mes compiled with gcc is at least twice at fast, and still pretty slow <deesix>It also happens that mes main task is generate and collect garbage (only half joking here). <janneke>at least it's 100% memory safe, and not "somewhat memory safe except when it really matters" :) <deesix>mes is as safe as any C program. <janneke>when i say mes is memory safe, of course i mean scheme code executed by mes <janneke>just like mes, the c program isn't slow, but scheme code executed by mes is <deesix>My point/joke that it uses (fast or slow) lots of time just in the garbage realm (as opposed to computation). <deesix>Which is why it eats memory bandwidth like crazy and machines with slow memory suffer for weeks. <janneke>this is exactly one of the areas a true lisp machine would really help <janneke>when i re-read sicp to get inspiration for mes, my laptop already had 16G of memory and i was inspired by the footnote: "This may not be true eventually, because memories may get large enough so that it would be impossible to run out of free memory in the lifetime of the computer. For example, there are about {3\cdot10^{13}} microseconds in a year, so if we were to ‘cons’ once per microsecond we would need about 10^{15} cells of <janneke>memory to build a machine that could operate for 30 years without running out of memory. That much memory seems absurdly large by today’s standards, but it is not physically impossible." <janneke>alas, we're not there yet today, mes still needs to collect garbage on most machines to compile tcc <ekaitz>janneke: but we could get nearer from that with a couple of very cool tricks that I have thought about already hehe <ekaitz>once we have the new interpreter we could work on it <janneke>it would be interesting to know how much memory we actually need/use at the moment <ekaitz>but still, we could try, mmap one terabyte and then let it fail <deesix>I'm pretty sure many here have seen the GC kick and kick and kick for Gs and Gs, nonstop. <aggi>janneke: it's remarkable criticism is directed towards tinycc first, instead of g++/binutils <aggi>of cause, there's some plausible reasoning to remove tinycc from live-bootstrap, (and replacing this with pnut-exe or whatever) <aggi>from another perspective, if you recognize the fact tinycc is the _only_ C-toolchain remaining which a complete GNUish/POSIX system-integration was confirmed with <aggi>don't know why tinycc and live-bootstrap didn't cooperate for tcc-head rebasing <aggi>i've noticed weird statements from both sides <aggi>and a third one, that was #gentoo who denied a re-integration for tcc-toolchain support <aggi>finally i've forked that too <aggi>and tinycc was important for another reasons, to verify which "free software" remained nowadays if there wasn't any g++/c++ involved <aggi>i've made it up until libsdl1/prboom/ffmpeg/mplayer/tetex/ghostscript etc... lots of things on framebuffer terminal <aggi>which is why, it's fascinating, a proposal to remove tinycc, _before_ an alternative for it existed <aggi>in any case, the confirmation with tinycc too established a BASELINE for another approaches with whichever C-toolchain might exist in the future <aggi>given where g++/binutils arrived at nowadays (pnut-exe is ~10000LoC in comparison) <stikonas>yaeh, mes is very heavy memorywise. That was especially noticeable on RISC-V visionfive2 board where bootstrapping tinycc takes weeks <stikonas>visionfive2 has fairly low memory bandwidth, probably just a few GB/s <stikonas>anyway,there is no single true bootstrap chain <stikonas>like I said before, people can just use different compoents <stikonas>you are definitely free to use tcc yourself <aggi>depends, if you want a GNU/Guix up and running, ther's no way to go with tcc <stikonas>and probably many other people will use it too <stikonas>well, tcc is not sufficient to bootstrap all the way to Guix <aggi>it's not specific to guix either <aggi>i've tested busybox last night <aggi>seems M2libc/mes-libc, do not suffice <aggi>wanted to hook in busybox-1.2 as early as possible <aggi>in combination with linux-tcc kernel, this provides full IPv4 networking very early in the dependency chain <deesix>stikonas, I have a note... visionfive2 uses LPDDR4 which is "up to" 400MB/s (that I guess best in theory). So... <deesix>But it's still a lot of memory usage. Like even hard to imagine. <stikonas>well, it's not reserved memory but I think a lot of bandwidth <stikonas>but I didn't look this much into mes internals <stikonas>to see how it's using it, i.e. reads or writes... <deesix>Yes, sorry, I meant bw usage. Whatever is allocated, over and over again. <lanodan>Oh TIL LPDDR has very different bandwidth compared to DDR for roughly equivalent clock/transfer rates <stikonas>visionfive2 is on the extreme end of slowliness <Googulator>LPDDR can be as narrow as 16 bits, while mainstream DDR (at least in the form of DIMM/SODIMM) is 16 bits wide per module <Googulator>but there's no upper limit - Apple M3 Ultra has a 1024-bit memory bus, and it's LPDDR <Googulator>AmpereOne and some AMD EPYCs have 768-bit buses per socket (1536-bit for a complete system), using DIMMs <Googulator>VisionFive2 probably went with the narrowest bus possible (16 bits), and possibly needs OS support for memory reclocking to actually get the full bandwidth of even that <Googulator>But during a typical Guix bootstrap on x86 using bootstrap-guile, gash/gash-utils are extremely slow, even though Mes isn't involved at all <Googulator>(this could be a different issue though, as it's slow even on machines with fast memory, unlike Mes) <stikonas>visionfive2 is somewhat disappointing that they abandoned upstreaming effort... <stikonas>though upstream support is good enough for bootstrapping <lanodan>Isn't everything but HDMI stuff upstreamed? <stikonas>yeah, so I guess good enough for bootstrapping <lanodan>Yeah although there's uart, that's mostly how I use mine when it's not ssh <lanodan>And I guess if you're not using the M.2 slot you can plug a GPU there <stikonas>GPU would probably have orders of magnitude more compute power... <lanodan>And probably be a lot bigger if it's not a basic one <stikonas>yeah, rebuilding those c99 tables is indeed about 2.5x slower on nyacc 3 compared to 1.00.2 <stikonas>with guile I guess it wouldn't be so noticeable <stikonas>ekaitz: do you know if you changes would help here? <ekaitz>what tables are those? something that has to be rebuilt per-file? <ekaitz>and the whole mes game is 2 times slower? or just that step? <ekaitz>my work at this point is not faster than old mes really <ekaitz>my macro expander is probably faster than the old one <stikonas>well, my guess was that nyacc 3 has better C11 support and needs execute more code <ekaitz>and the evaluator will become faster <ekaitz>my goal was to make it 2x faster <stikonas>it used to be ~5 minutes on my laptop, now about 14 <stikonas>well, even 2x would mostly get us back to the same speed <ekaitz>we still would need to make mescc be able to compile C11, right? <stikonas>hmm, AI thinks mes-m2 is more than 100 time slower than guile. And mes build with gcc is twice as fast as mes-m2 <ekaitz>heh all this is not imminent then <ekaitz>comparing to that article, both upstreamed mes and the new core are faster, which is cool <stikonas>AI probably printed numbers from your blog :D <stikonas>oh well, they scrape everything these days... <ekaitz>better asking me than AI for this, because those are very tricky numbers <ekaitz>you can see how guile there relies on having everything pre-compiled and cached to be that fast <ekaitz>the numbers I really like are chibi's <ekaitz>there are too many things I still need to improve in the interpreter :( <mwette>hmm. on my laptop, just generating c99 tables takes 5.0 s for 1.00 and 6.9 s for 3.04 <mwette>gram.txt files (list of all productions and states) are 21889 lines for 1.00 and 24697 lines for 3.04 <mwette>time cost could be correctness of parser :) <mwette>but lalr processing is certainly not linear in number of rules (432 for 1.00 and 473 for 3.04) <stikonas>I guess because guile pre-compiles it as ekaitz said <stikonas>I think that roughly agrees with those benchmarks ekaitz posted <mwette>exactly, so if you went to guile's lalr, you'd be sad, I think, because that does not seem to save tables in files.