IRC channel logs

2021-09-12.log

back to list of logs

<theruran>thanks xentrac! I read it all and I think it's an interesting idea for bootstrapping! same with CPLDs. there are free designs of CPLD somewhere? can they be scaled up easily?
<xentrac>theruran: oh cool! glad you liked it! yes, all designs of CPLD are free, unless someone has designed new ones since 02011, which seems unlikely. but in any case the first 23 years of CPLD designs are all free
<xentrac>at least in the US, though possibly elsewhere 02006 is the cutoff date
<xentrac>there are individual CPLDs that you could probably fit a 25MHz CPU into: https://www.digikey.com/en/products/filter/embedded-cplds-complex-programmable-logic-devices/695?s=N4IgjCBcpgrB0QAcAuUQEkB2KCmBzXAJxABoQBbASy3TAAZH6QBfcgWgCYpQBjKFEQCuucgHsoAbRCwQAXTYh2ATh4h%2BkQSPFTw8looBsaqgBN07BhHKp0ZEAEcUATzr2XSXOgCGAZ34GQA
<oriansj>xentrac: I would be happy with a 100khz CPU
<oriansj>if it was fully free
<xentrac>for example the Xilinx XC2C512 CoolRunner-II, which looks like it was introduced in 02002 and costs US$85: https://www.digikey.com/en/products/detail/xilinx-inc/XC2C512-10FGG324C/966596
<xentrac>the XC2C512 is fully free in the sense that there's nothing secret about it and there are no legal restrictions on copying it, but it's manufactured in 0.18μm, which might make auditing it with die photos a little difficult
<xentrac>and, being a CPLD, it's pretty low on memory. if I'm reading the datasheet right it contains 512 bits of memory (not counting the configuration memory)
<xentrac>oh, no, there are also 32 16-bit up/down counters
<xentrac>> .com.
<xentrac>> Third party development tools include synthesis tools from Cadence, Exemplar, Mentor Graphics, Synplicity, and Synopsys.
<xentrac>oops
<xentrac>.com.
<xentrac>> Third party development tools include synthesis tools from Cadence, Exemplar, Mentor Graphics, Synplicity, and Synopsys.
<xentrac>ugh
<xentrac>> Xilinx CoolRunner-II CPLDs are supported by all configurations of Xilinx standard release development software as well as the freely available ISE WebPACK software available from www.xilinx.com.
<xentrac>that was the missing line
<xentrac>so, a world of difference from the FPGA "you must use our software or else" world
<xentrac>still, I'm having a hard time finding the specs for the bitstream format... but I trust it's out there
<xentrac>looks like azonenberg had to reverse-engineer it: https://recon.cx/2015/slides/recon2015-18-andrew-zonenberg-From-Silicon-to-Compiler.pdf
<oriansj>xentrac: I don't trust until I see a properly Free software licensed tool that provides the assumed functionality.
<xentrac>that is as it should be!
<oriansj>I know tedious boring shit is the bits that people never do without some serious reasons to do so.
<oriansj>and that software sounds tedious to write, so it needs something like a paycheck or a person reason before someone would do that work.
<xentrac>seems like it does exist: https://robertou.com/unofficial-open-source-place-and-route-for-xilinx-coolrunner-ii-cplds.html
<xentrac>(expired SSL cert)
<xentrac>azonenberg open-sourced his software as libcrowbar
<xentrac>oh also it looks like Claire Wolf did use her yosys to map her PicoRV32 to TTL chips: https://old.reddit.com/r/FPGA/comments/3kn5qu/open_source_fpgacpldpla/
<xentrac>but because she was using only the 7400, 7402, 7404, and 7474, the design requires 4846 chips, which would be half a square meter of PCB, which is actually pretty close to uh
<xentrac> https://hackaday.io/project/178826-pineapple-one which was designed in a more sensible way
<xentrac>(500 kHz, 512 KiB RAM, 200×150 monochrome VGA output, RV32 (I, I think))
<xentrac>for example, not using TTL :)
<xentrac>(using CMOS instead of TTL doesn't affect the chip count much; using higher levels of integration is what accounts for that. CMOS just gives you better noise immunity, much lower power consumption, and higher speed)
<stikonas>yes, pineapple-one sounds quite good for bootstrapping, it was mentioned here before
<xentrac>sorry, I didn't remember
<stikonas>well, maybe you were not online at those hours...
<stikonas>I'm just saying that somebody discussed it here
<stikonas>500 kHz should be enough for bootstrapping up to M2-Planet + tools
<xentrac>well
<xentrac>unless you need to debug something ;)
<oriansj>stikonas: unfortunately the hex2_x86.hex1 change you suggested https://paste.debian.net/1211419/ did not have the improvement in performance hoped for.
<oriansj>we have only sped up from 18:07.60 to 15:13.74 (a good improvement 16% speed improvement though)
<oriansj>./hex2-0 hold M2
<oriansj>went from 6:31.76 down to 5:30.73
<oriansj>xentrac: we can develope on big powerful machines, a 500khz machine would only be used for running the bootstrap (or in the event we find a difference, isolating the difference for inspection)
<oriansj>makes me wonder if I should malloc the table onto the HEAP
<oriansj>probably making scratch part of the heap will probably help with performance too stikonas
<xentrac>oriansj: the bootstrap machine has to be able to operate autonomously; it can't be slaved to an untrusted machine the way an Arduino is slaved to your laptop
<xentrac>the flow of code updates needs to be *from* the bootstrap machine *to* the untrusted machines, not vice versa
<Hagfish>great point
<oriansj>xentrac: yes hence why SET exists and more is written in hex0
<xentrac>so you have to be able to edit on it. 500kHz is barely usable
<xentrac>SET?
<oriansj> https://github.com/oriansj/stage0/blob/master/stage1/SET.s
<oriansj>shitty expensive typewriter
<oriansj>it is a line editor
<xentrac>aha, I see
<oriansj>that can run at 5hz speeds and still be responsive
<xentrac>nice!
<xentrac>yeah, my experience is that 0.3 DMIPS is about the minimum for a responsive screen editor
<oriansj>but editing a line however is quite a bad user experience because you have to type it correctly or do it all again.
<xentrac>have you seen ae.c, Ant's Editor?
<xentrac>Anthony Howe's editor
<xentrac> https://groups.google.com/forum/#!topic/comp.editors/SID2De137wY
<xentrac>it's quite a bit bigger though, about 6K
<xentrac>(compiled for amd64!)
<xentrac>it's a reasonably usable implementation of vi
<oriansj>no but SET is good enough for me and it is only 1072bytes
<xentrac>well, as you point out, having to type an entire line correctly or do it all again is quite a bad user experience
<xentrac>that will result in code updates flowing in the fatal direction
<oriansj>better than having to rewrite the entire file like in the hex0-monitor
<xentrac>true. I have written programs with COPY CON OCTAL.COM
<xentrac>it's a pain
<oriansj>hex0-monitor (not for anything larger than what is needed for SET)
<oriansj>and SET is good enough for cc_x86.M1
<xentrac>it's not good enough for GCC
<oriansj>it would be interesting to see someone do a vi for knight bare-metal
<oriansj>as ltermcap and lcurses are certainly not available
<oriansj>xentrac: 500hz isn't good enough for MesCC but at that point we probably need something with enough power for a real set of tools
<xentrac>yeah, ae.c does depend on curses
<oriansj>and TCC can certainly build quite a bit
<oriansj>So honestly we only need good enough to inspect source code up to TCC max
<xentrac>initscr, stdscr, addch, mvaddstr, clrtobot, move, refresh endwin
<oriansj>and https://github.com/oriansj/stage0/blob/master/stage1/more.s is good enough for reading of code
<xentrac>well, being able to inspect anything is good, and being able to inspect more things is better
<xentrac>for reading code you really need ctags
<oriansj>xentrac: ctags and vim will not work well at 500khz
<oriansj>also this is still bare metal, so no operating system to run ctags or vim
<xentrac>they'll be marginally usable. they don't need an OS
<xentrac>I mean Turbo Pascal ran acceptably fast without an OS on a 2MHz Z80, which is about 0.5 DMIPS, same as a 500kHz RISC-V
<xentrac>the IDE
<oriansj>in their current form, they absolutely do. given a sufficient large amount of changes, no it would be possible to make them run on bare metal and read/write the filesystem
<xentrac>remember ctags dates from 01978
<xentrac>the first version was a shell script that invoked ed
<oriansj>xentrac: nope, don't use ctags personally
<xentrac>do you mean you don't use anything that gives you the facility to jump to the definition of the function your cursor is on a call to?
<xentrac>or that you do, but it's not called ctags?
<oriansj>C-s works just fine
<xentrac>yes, if it's in the same file
<xentrac>and your machine is fast enough to not need an index
<xentrac>I think it's somewhat dispensable for writing code, but not for understanding code written by somebody else
<oriansj>I like to follow the flow to understand other people's code
<xentrac>you mean, the control flow? so what do you do when you reach a call to a function you're not familiar with?
<oriansj>grep
<xentrac>right :)
<xentrac>ctags is like a precomputed grep
<oriansj>and probably quite useful
<xentrac>try it, etags $(find . -name '*.[ch]')
<xentrac>then use M-. in Emacs
<xentrac>most of the complexity of curses comes from the need to optimize updates over low-baud-rate terminal links. repainting a 2-kilobyte terminal text buffer in RAM from scratch is fast enough that even at 0.3 DMIPS it's only slightly annoying, about 10-30 milliseconds
<xentrac>doing it over a 2400-baud serial line, OTOH, takes 8000 milliseconds, which is about two orders of magnitude too high for the response to a keystroke
<xentrac>the back button for tags in Emacs is M-*
<xentrac>M-. to follow a link, M-* to go back. vh works the same way, so does ctags (except in old vi you can only have one file open at a time, so you'd better save first)
<oriansj>trial on M2-Planet: results ewwww
<xentrac>haha, what happened?
<oriansj>it seems to have confused tests with actual code
<xentrac>oh, maybe you should rebuild the TAGS file excluding the tests directory
<oriansj>I am quite certain if I take the time to learn the tool properly and use it properly it is lovely
<xentrac>easiest way is to start typing `find *`, type C-x * to expand out the *, then delete the test directory from the list
<xentrac>rather than find . -name t -type d -prune -o -name '*.[ch]'
<oriansj>xentrac: like I said, if I take the time to learn and use it properly I am certain it is probably useful
<oriansj>doesn't work at all in hex? or M? files though
<xentrac>which is error-prone. in fact I screwed it up: find . -name t -type d -prune -o -name '*.[ch]' -print
<xentrac>yeah, that's true! if you write your own programming language ctags/etags won't work with it out of the box
<oriansj>xentrac: perhaps a better idea is just use emacs to add to the etags as you hit files in a project
<oriansj>also it would require a separate tag file for each test as well
<xentrac>like, with imenu?
<oriansj>well jumping out of the emacs flow is less than ideal
<xentrac>right, normally I build the TAGS file once when I first start messing with a codebase
<xentrac>thereafter M-. consults it and silently warps me to where I want to go
<oriansj>fair and it certainly makes jumping around fast and easy
<xentrac>I thought you meant something like "use imenu's buffer-indexing code to figure out what definitions are in a file because imenu already has M3 support"
<oriansj>well M3 is just a standard gas assembler
<xentrac>well, M1, then
<xentrac>yeah, which is what you want when you're trying to audit a codebase: it's important to quickly be able to figure out what you can assume about the code that's being called from a callsite
<xentrac>before ctags people would use xrefs generators for their printouts
<xentrac>sometimes the compiler would automatically append xrefs by default to the listings it generated
<oriansj>now an automatic xref generator for code might be nice to help me build notes when starting a new code base.
<xentrac>that's what ctags is
<xentrac>you can run ctags -x to get a more human-readable output format
<oriansj>it truncates the input variables section (which would actually be useful)
<xentrac>hmm, not for me; are you running Exuberant Ctags?
<xentrac>or do you just mean that the text quoted from the declaration is cut off at the end of the line?
<oriansj>ctags (GNU Emacs 27.2)
<oriansj>it is cut off after the ( like: struct type* promote_type(
<oriansj>even though the actual like of source code is: struct type* promote_type(struct type* a, struct type* b)
<xentrac>yeah, Emacs ctags does that. Exuberant Ctags doesn't, which has its pluses and minuses
<xentrac>(but Exuberant Ctags is happy to generate Emacs TAGS files if invoked with -e or as the etags command)
<oriansj>xentrac: is it in guix yet?
<xentrac>probably? I mean Vim ships wit hit
<xentrac>or used to anyway
<oriansj>I'm a vi (not vim, bare minimal vi) on server and emacs on local sort of guy.
<xentrac>well, vi has shipped with ctags since 01978 ;)
<xentrac>hmm, maybe not? it's not on https://guix.gnu.org/packages/E/page/17/ or https://guix.gnu.org/packages/C/page/8/
<xentrac>I think Roland McGrath added tags support to Emacs in 01985
<xentrac>§32.3 of the Emacs user manual is about tag tables
<xentrac>part of §32 on "maintaining large programs", which nowadays are just called "programs" ;)
<xentrac>I think diff is also an essential tool for inspecting software version histories, btw
<oriansj>umm no. vi doesn't come with ctags
<oriansj>it has view and ex but no ctags
<oriansj>(and a man page)
<oriansj>you can verify yourself: https://packages.debian.org/buster/amd64/nvi/download
<xentrac>presumably in Debian it's been split into a separate package (several actually)
<xentrac>but that was not the case in 01978
<oriansj>xentrac: I wasn't even alive in 01978 or for even years after it
<oriansj>So I am using tools as they are available now. ctags never was part of the programming experience for me. I can see its attraction and ways that it could make a certain style of development much more productive.
<xentrac>well, both exuberant ctags and Emacs ctags/etags are in Debian
<oriansj>so thank you xentrac for introducing me to a new tool to add to my toolkit
<xentrac>sure :)
<oriansj>It certainly will improve my Java development in Emacs
<xentrac>it's not that great at Java because so many methods have the same name
<xentrac>I think Emacs JDE has a smarter version
<xentrac>it comes with a program called jtags
<xentrac>but for Java I just use Eclipse ;)
<oriansj>but it is soooo slow
<xentrac>yeah
<xentrac> https://ignatyev-dev.blogspot.com/2012/10/java-development-with-emacs.html recommends using GNU GLOBAL to generate tags files for Java
<xentrac>which is in guix: https://guix.gnu.org/en/packages/global-6.6.7/
<oriansj>well most emacs tools are in guix because guix is a very emacs heavy community
<xentrac>it's not really an emacs tool
<xentrac>I mean it does work with Emacs but it kind of works with anything
<oriansj>xentrac: fair
<oriansj>you also have to remember there is this weird generation gap in programming where once people started teaching with IDEs a great many useful tools and ways of working (like vim and emacs) were entirely dropped on the floor.
<xentrac>I think of ctags as being the major feature of IDEs, really
<xentrac>although jumping to compile errors (C-x `) is nice too
<xentrac>haha, I just found out that the first Berkeley RISC also ran at 500kHz: https://en.wikipedia.org/wiki/Berkeley_RISC#RISC_I
<oriansj>well one doesn't need much power to get very far when one uses simple tools
<xentrac>oh actually maybe it was 1MHz, because it was a 2-stage pipeline
<theruran>I want to write a bootstrapping compiler for Ada in Scheme. where should I start? "Hello, World"?
<theruran>thinking of using Guile's PEG parser tools
<theruran>just wondering if you all had any words of advice for someone who's never written a compiler before
<oriansj>theruran: the first word of advice is no programming language is magic.
<oriansj>second word of advice is try return 42 first
<xentrac>hmm, does Ada have a built-in way to return exit codes?
<oriansj>have tests for every primitive you support and slowly grow that list
<xentrac>PEGs will make your life a lot easier than things like lex and yacc, which in turn are a little easier than writing a parser by hand
<xentrac>not familiar with Guile's tools in particular
<xentrac>oh, maybe start with the backend first
<xentrac>that way you have a compiler from the beginning, instead of a parser that doesn't compile
<xentrac>though oriansj may be a more trustworthy source of advice on this count
<oriansj>I'd put janneke as the expert on writing compilers in scheme.
<oriansj>but output assembly instead of trying to directly write bytes unless you like spending time figuring out that you made a mistake in outputing bytes again.
<theruran>Mes outputs M1 assembly?
<oriansj>non-optimized output is just a simple state machine, optimized output however is a great deal more complex
<oriansj>theruran: correct MesCC produces a M1 assembly file then calls M1, bloodelf and hex2 to convert into a running binary
<theruran>isn't outputting M1 better for our purposes than something else?
<theruran>Guile exposes its compiler infrastructure, so you could output to that IR
<oriansj>theruran: I strongly suggest standard GAS assembly output rather than M1 for multiple reasons.
<oriansj>most of which about saving you considerable effort and time.
<theruran>A.15 Package Ada.Command_Line
<theruran>procedure Set_Exit_Status (Code : in Exit_Status);
<theruran>oriansj, got it. thanks for the advice!
<theruran>what does it mean to start with the backend first? the emit_* functions for each GAS assembly instruction?
<theruran>or the most common ones, anyway
<oriansj>nope
<theruran>Ada types?
<oriansj>start with just int
<oriansj>compile an Ada function that just returns 42 to whatever called it
<theruran>starting with a handmade AST?
<oriansj>whatever seems like more fun.
<theruran>good advice
<oriansj>if you want simple, you can always roll your own: https://github.com/oriansj/M2-Planet/blob/master/cc_reader.c
<oriansj>which is just a list of tokens made from the source file
<oriansj>then slowly grow your state machine: https://github.com/oriansj/stage0/blob/master/stage2/High_level_prototypes/cc_x86/cc_core.c
<oriansj>or if you want do things the proper way with a proper grammer
<oriansj>and a proper AST
<oriansj>Just do what seems like something that better matches your development style
<theruran>hoping to find programming fun again
<theruran>what about studying GNAT's AST? would be nice to see some diagrams of that
<oriansj>theruran: There are not more answers I can give you because compilers grow to reflect the people working on them.
<theruran>:)
<theruran>I found this anyway: https://en.m.wikibooks.org/wiki/X86_Assembly/GAS_Syntax
<xentrac>I think Intel syntax is also reasonable, maybe slightly less annoying than gas syntax. and gas supports it
<xentrac>Guile's IR sounds like it might be an interesting target! can you compile it to a standalone binary though?
<xentrac>by "start with the backend first" I meant "write some code that outputs a program in machine language or assembly before writing a parser"
<xentrac>I mean in Guile you can type in an abstract syntax tree as an S-expression if you want
<xentrac>but "whatever seems like more fun" may be better advice :)
<theruran>Guile's CPS language compiles to the VM bytecode
<theruran>I think the Guile 3's JIT compiles to native code using Lightening
<theruran>bytecode to native code
<xentrac>awesome. the VM bytecode might be a lot easier to generate than assembly code too, though I haven't looked at it
<theruran>here's what "return 42" looks like using GNAT
<theruran> https://www.irccloud.com/pastebin/Tp3YjShz/
<theruran>using -O2 -S -c. not sure if there are other flags I should use
<theruran>I see. it JMPs to the standard Ada library instead of RETs
<theruran>in Ada, you don't need to call the main function "main"
<oriansj>theruran: neither do you in any real compiled language if you are willing to write a little assembly
<oriansj>but yes loading a register with 42 and doing a syscall for exit is probably the correct way to do it
<theruran>the GNAT manual is pretty good!
<theruran>xentrac: AOT compilation to native code emission is not yet implemented in Guile's compiler
<theruran>The next step is probably to add ahead-of-time native code emission to the back-end of the compiler written in Scheme, to take advantage of the opportunity to do global register allocation and instruction selection. Once this is working, it can allow Guile to experiment with speculative optimizations in Scheme as well. See Extending the Compiler, for more on future directions.
<theruran> https://www.gnu.org/software/guile/manual/html_node/Just_002dIn_002dTime-Native-Code.html#Just_002dIn_002dTime-Native-Code
<xentrac>so if you go that route you probably don't want to do a syscall for exit; you probably want to return 42 to the Scheme interpreter that your compiled Ada code is being called from
<stikonas>oriansj: your x86 change gives me significantly better improvements
<stikonas>it's way more than 16%
<stikonas>building M0_x86.hex2 goes from "real 0m9,548s" to "real 0m1,853s"
<stikonas>basically 4 times faster
<stikonas>maybe you started hitting some other bottleneck (sdcard?)
<stikonas>Risc-V gains are probably better because I (from the very beginning) kept file descriptors in registers rather than memory (both input/output file)
<stikonas>and also other things (like Flag)
<stikonas>only scratch (&table) was kept in memory in risc-v
<stikonas>which I could easily do because risc-v has loads of registers
<stikonas>I guess it's not so easy on x86
<fedorafan>yo
<oriansj>stikonas: I am glad you got a big benefit for qemu for x86 and yes the SDcard might be the new bottleneck but I'll have to do some testing with a RAMdisk to verify that
<oriansj>fedorafan: greetings
<fedorafan>what is this bootstrak i often heard of
<opal>a typo of bootstrap
<fedorafan>and what is that language?
<fedorafan>framework
<opal>it's a project not a framework
<opal>read the site in /topic
<fedorafan>im here to ask not to google
<oriansj>fedorafan: this is about software bootstrapping not the javascript framework called
<opal>everyone in here will tell you to read the topic
<fedorafan>ah its js framework thanks
<opal>there is no "googling" involved in the topic
<fedorafan>yeah if im further interested i will read the topic
<fedorafan>of course
<oriansj>fedorafan: waste not time of real bootstrappers
<fedorafan>i ignore you
<fedorafan>fucker
<opal>what the fuck
<opal>im here to ask the same question everyone else asks, not to check the topic
<opal>not to check the public logs, because evidently this channel has those
<opal>just spoonfeed me
<oriansj>it is very stupid to insult one with ops
<opal>sounded like they were more pissed at me
<oriansj>doesn't matter who they said it to. That behavior is entirely unacceptable
***ChanServ sets mode: +o oriansj
<oriansj>everyone here is a person contributing to solving a problem that matters to all of us and thus deserves basic consideration and respect.
***oriansj sets mode: +b fedorafan!*@*
<oriansj>now we ignore them *FOREVER*
<oriansj>or until they grow up, whichever comes first
<stikonas>argh, my riscv64 M0 has a small issue
<stikonas>strange...
<stikonas>fails to hexify strings with numbers
<stikonas>shouldn't be hard to fix...
<oriansj>stikonas: are you talking about raw string support in M0?
<stikonas[m]>Yes
<stikonas[m]>It bailed out on alphas string in cc_*
<oriansj>well it shouldn't care about anything but if it is a close " or just another unpack
<oriansj>or if you null terminated, zero
<oriansj>RS1_A2 RS2_A0 @24 SD ??
<oriansj>oh yeah risc-v doesn't have actual proper stack ops
***oriansj sets mode: +q fedorafan!*@*
***oriansj sets mode: -b fedorafan!*@*
<stikonas>oriansj: after some debugging I think processing strings is fine
<stikonas>the crash is later in Eval Immediates
<stikonas>it's actually not a crash but I'm hitting Fail branch in express_number function
<stikonas>so the problem is more or less clear
<stikonas>now let's do the fix
<oriansj>well that should only happen if the number isn't prefixed by one of "~!@$"
<oriansj>are you still hitting that function with %-1 ??
<stikonas>I was hittign that function with "9" in test.M1 file
<stikonas>well, it's a bug in M0 only, M1 deals with it fine
<stikonas>I think oversight in Eval_Immediates
<stikonas>probably got rid of too many checks
<oriansj>So the function prior is miss identifying
<stikonas>exactly
<stikonas>should be easy to fix
<stikonas>well, I did test M0 with all files in Development/*.M1 but none of those had this
<oriansj>well a raw 9 shouldn't appear in M1 anyway
<oriansj>it should either be '9' or "9" or DEFINE thing 9 or %9
<oriansj>if it was in a define it shouldn't even be looked at
<oriansj>only case would have been !9, ~9, @9 or %9 that should have hit that code
<stikonas>well, it's just "9" (3 bytes)
<stikonas>well, plus LF
<stikonas>anyway, I should have a fix soonish
<oriansj>I->EXPRESSION should be NULL at that point only for immediates and invalids
<oriansj>so if I->EXPRESSION isn't null you should be skipping to next as well
<oriansj>and "9" should absolutely not have I->EXPRESSION at this step (as strings are processed before immediates)
<stikonas>oh, it's probably I->EXPRESSION==NULL check that is missing
<stikonas>indeed
<stikonas>that's 3 line fix...
<oriansj>nicely done stikonas
<stikonas>or actually just 2 instructions in riscv
<stikonas>well, you last comment helped me quite a bit to find issue quicker
<stikonas>ok, fixed
<stikonas> https://github.com/oriansj/stage0-posix/pull/44
<oriansj>merged
<xentrac>stikonas: woohoo