<muurkha>the particular ML features that make ML better than C for compilers are efficient automatic memory management (usually GC), strong static typing with sum types, statically checked pattern matching, implementations that generate efficient code, and duck typing in the form of higher-order functions <muurkha>of these Scheme has the first and the last <muurkha>I'm inclined to think that the *implicitness* of typing in ML and Scheme is also a benefit, but I think there are people who would disagree with that, and they might be right <muurkha>I feel like other aspects of parametric polymorphism like the ability to add your own safe polymorphic containers are maybe not as important in the context of compilers <oriansj>well polymorphism never seems like a real good idea when you are working in a problem space of meaningful results <muurkha>I don't know how to interpret that statement as something that could be true; can you elaborate? :) <muurkha>parametric polymorphism means that you can implement, for example, quicksort, binary search, a hash table, or a B-tree once, and then use it on many different types of objects, rather than having to implement a separate quicksort or hash table for each type of object <oriansj>a function or a data type can be written generically so that it can handle values identically without depending on their type; but types REALLY matter for most specialized operations. <oriansj>ok sort {name: foo; age: 1; ssn: 1234567} {name: bar; age: 10; ssn: 2345678} <oriansj>and now consider a language where functions can be passed. <oriansj>sort(a, b, myfunc); that is just plain C no polymorphic needed <oriansj>but maybe because my dream language is based on the idea that every bit of the language becomes part of the source code space. <oriansj>DEFINE (INT64 a) + (INT64 b) AS LAMBDA RETURNS INT64 .. sort of thing <drakonis>for interesting bits related to the language <drakonis>your dream language is going to eventually release <oriansj>well it would be a simple language to implement. There is only 3 keywords and a list of types <drakonis>though you gotta implement a standard library, innit? <oriansj>LAMBDA is defined in terms of TYPE and ASM <oriansj>just like stacks, iteration and even the definition of LAMBDA <oriansj>and the meaning of RETURNS gets kinda of interesting when you can change the number of arguments returned (literally forcing the programmer to manually deal with the impossible compiler problem) <oriansj>(which is already done in assembly programs like cc_*) <muurkha>sort(a, b, myfunc) is definitely polymorphic. the polymorphism is the reason why qsort() and bsearch() in C use void pointers <muurkha>that's a good example of how to do duck typing in the form of higher-order functions; the problem is that in C you don't get the duck typing, so you have to just turn the type system off entirely <muurkha>incidentally, I thought the Golang approach to this problem was interesting <muurkha>Golang's sort.Interface consists of Len() int, Less(i, j int) bool, and Swap(i, j int) <muurkha>so it's a type for the whole container, not the individual elements <oriansj>well think about this DEFINE ANY AS TYPE {ENUMERATION of types}; (then the programmer can redefine ANY to include their custom types) and then DEFINE SORT (ANY A, ANY B, COMPARE) RETURNS ANY ... <oriansj>and DEFINE FOO AS COMPARE (int64 a, int64 b) <muurkha>sorting is a good example where it's *really* useful to be able to use a debugged reasonably efficient (and necessarily polymorphic) linearithmic sort function from the standard library rather than weighing whether it's worth the effort in this case of implementing heapsort yet again or if you should just go with insertion sort <muurkha>or whether quicksort is worth the risk of worst-case quadratic behavior (Golang's standard unstable sort is introsort to avoid this) <muurkha>but I would argue that, if for whatever reason you're stuck writing a compiler in Pascal, the amount of extra work from manually implementing potentially-polymorphic things like insertion sort N times is going to be a small fraction of the time you spend on the compiler <muurkha>(I mean standard Pascal, not the more reasonable vendor-specific extensions that people usually use) <oriansj>well insert sort for me tends to be shorter than quick sort <muurkha>it absolutely is shorter, and it is often faster <muurkha>I was going to say "but it's still not as short as a call to a standard library routine" but in the case of Golang's sort.Interface that might not actually be true ;) <oriansj>hmmm insert sort in C is 8 lines if one keeps it clean; <muurkha> if (a[j-1] > a[j]) swap(&a[j], &a[j-1]); <oriansj>DEFINE mycompare (mytype a, mytype b) returns bool {return a->name < b->name}; SORT (a, b, mycompare) <muurkha>I guess that presumes the implementation of swap <muurkha>right, invoking a purely comparator-based interface can be a bit terser, especially if you have lambdas <muurkha>but Golang's sort.Interface requires you to implement Len, Less, and Swap <oriansj>well copy rather than update removes the meaning of swap for sorting <oriansj>if you are doing a sort on an array of type int64; you just create another new array of type int64 and return that <oriansj>there is no swap, only values copied <muurkha>do you mean that you can implement a sort without swaps? that's certainly true <muurkha>insertion sort in particular can also be significantly faster if you do it that way <muurkha>and mergesort doesn't really benefit from swap <muurkha>a nice thing about heapsort, though, is that it's very predictable <oriansj>also in retrospect if ANY is defined to be expanded to iterate the function over all possible types and then pruned to only those combinations of those used; you enable some fun optimizations <muurkha>it never fails because you don't have enough memory, it never takes more time than it usually does because the input is in a different order <muurkha>yeah, there are a lot of interesting whole-program optimizations like that that are rarely used <oriansj>muurkha: no quantum-bogosort attached to destory universe device? <muurkha>not in my standard library, no. universe-destroying devices are not a portable solution because they don't work in universes obeying the Copenhagen interpretation rather than MWI <muurkha>unlike parametric polymorphism, automatic memory management has a pervasive simplifying effect on your code, because it allows you to pass around arbitrary data structures as easily as ints <muurkha>and pattern-matching simplifies almost every part of a compiler <oriansj>and Pointers despite being *VERY* powerful tend to be misused so easily <oriansj>so I am torn between 2 options: no pointers but Garbage collection and safety or pointers and absolute insane levels of freedom in the language. <muurkha>by "pointers" do you mean pointer arithmetic? or just shared references to mutable objects, as in Scheme? <oriansj>I mean real pointers in the crazy sense <oriansj>where the pointer could point in the middle of objects and arbitrary byte addresses <muurkha>well, there are several different kinds of language semantics where pointers can point into the middle of objects and arbitrary byte addresses <muurkha>one is the C/assembly approach, where you can freely convert between pointers and integers <oriansj>and could be objects or functions or parts of the stack <muurkha>another is the Golang approach, where you can't do that but you can embed objects inside other objects and then take pointers to them <oriansj>I was thinking the assembly approach (much more free than the C restrictions) <muurkha>maybe your options are best as two different levels in the same system, rather than alterantives? <oriansj>well rust is already trying to do that to a degree <muurkha>rust is the only language I know (other than Tcl and Bash) attempting to eliminate pointers in the sense of "shared references to mutable objects" <muurkha>which I think is a really interesting experiment, and seems to be bearing some very interesting fruit <muurkha>zig isn't attempting to eliminate pointers in the sense of "shared references to mutable objects", is it? <muurkha>but AFAIK it doesn't have the particular feature I was talking about, which is fairly central to Rust <oriansj>used cryptsetup to make /dev/sda1 into a luks volume but on init cryptsetup says /dev/sda1 is not a valid LUKS device <muurkha>if you cryptsetup luksDump /dev/sda1 from a livecd pendrive, what does it say? <oriansj>so it is valid but it looks like I forgot to setup mount -t devtmpfs none /dev on my initramfs <muurkha>I lost track of how devfs stuff worked around the beginning of systemd <muurkha>oriansj: Graydon's presentation mentions Turbo Pascal and Aztec C as examples of Variation #7 <muurkha>TP I'm pretty sure was written in assembly; not sure about Aztec C <muurkha>also he mentions something called 8cc <muurkha>"6'740 lines of C, self-hosting, compiles to ~110kb via clang, 220kb via self. Don't have to use assembly to get this small! Quite readable and simple. Works." <oriansj>well doing C compilers is pretty easy, even in assembly <muurkha>which David A. Wheeler's 'SLOCCount' tells me is 9'232 lines of C <muurkha>ah, the author is the original author of lld, which is why his name seemed familiar <oriansj>smart and capable people, tend to produce multiple interesting things in their lives <oriansj>if nothing else, this gentoo encrypted / and /boot has forced me to memorize the mount -t proc none proc; mount --rbind /sys sys; mount --rbind /dev dev; mount --bind /run run; chroot . pattern <oriansj>well I got it to the asking for luks password <oriansj>now I just get a device mapper error: reload ioctl on failed: invalid argument <oriansj>muurkha: well slices would be nice but I would have to say the biggest mistake is probably & (address of) because without it C would have been forced to a better more strict language <oriansj>well if I add to package.use sys-kernel/genkernel cryptsetup; I hit the linux-firmware being masked. So how would I get around that? <oriansj>and I mean in regards to not needing to use binary blobs <sam_>(although I'd still recommend not using genkernel) <oriansj>sam_: ok, I can avoid genkernel if I knew what the better way would be <oriansj>as I am willing to do a manual build and a hand rolled initramfs if I need <oriansj>and that will work with the current issue I am seeing with the lack of the dm_crypt module in the kernel I built? <sam_>I don't see why it shouldn't at all -- the reason I in particular dislike genkernel is that it makes it way too hard to actually use your own config <sam_>it ends up overwriting it evne when you think it wouldn't <oriansj>I can work with hard; I just need the process to be reliable enough <sam_>I would say it's not very reliable <sam_>I mean, some people like it, but it's not something I recommend for a reason <sam_>my experience has been that it has confusing UX and it's not as well maintained as it ought to be <muurkha>oriansj: & definitely constrains the compilation approaches applicable to C and can be bug-prone <muurkha>I think most of its uses are covered by Pascal-style var parameters: self pointers, state machines, multiple returns, in-out parameters, that kind of thing <muurkha>maybe paradigmatic is tcc's TokenString <muurkha> t = next_argstream(nested_list, &ws_str); <muurkha>but that can be handled just as well by heap-allocating it, which is what you'd do in Scheme <muurkha>somewhat more tricky is when you want to pass a mutable reference to a field to a subroutine; the simplest example is scanf: <muurkha>(void)sscanf(departure, "%02u-%02uT%02u:%02uZ", &ais->type6.dac1fid12.lmonth, ...); <oriansj>it literally isn't used once in M2-Planet or mescc-tools <muurkha>but a more compelling example, I think, is the sort of use you see in Dear Imgui, <oriansj>muurkha: but is that really actually a good idea <muurkha>where you can ImGui::SliderFloat for any arbitrary floating-point field <muurkha>that both reads and writes the field <muurkha>gpsd does something similar for its JSON serialization: <muurkha> {"lon", t_real, .addr.real = &gpsdata->fix.longitude, .dflt.real = NAN}, <muurkha> {"lat", t_real, .addr.real = &gpsdata->fix.latitude, .dflt.real = NAN}, <oriansj>and doing float a; ... ; a = SliderFloat(a); would achieve the same <muurkha>well, clearly & isn't necessary for Turing-completeness! <oriansj>and by removing it, you are forced to be more explicit <muurkha>but by removing it, the code is more bug-prone in that case <muurkha>because you can accidentally discard the return value, or store it somewhere else than in a <oriansj>muurkha: well sometimes b = SliderFloat(a); is actually what you want and SliderFloat(a); can easily be a compiler error <muurkha>no, with an IMGUI toolkit b = SliderFloat(a); is never what you want <muurkha>because on the next screen update the slider will still be at the same place it was before even though you dragged it <oriansj>muurkha: yes; sometimes you want to be a bad UI experience <muurkha>well, you could set a back to its previous value explicitly if you wanted that <oriansj>much like tetris which only returns the worst possible piece evertime <muurkha>Pascal-style var parameters handle cases like that pretty well, but maybe not the gpsd example <muurkha>occasionally a pointer to a pointer is handy for linked-list walking code. like, if you have a hash table with separate chaining, you want to follow the linked list chain from the appropriate hash bucket until you find a null pointer, but then what you do depends on whether you're inserting or just searching <muurkha>returning a pointer to the pointer (which might be an array element or a field of a linked-list node) is convenient in that case; it keeps you from having to write the function twice <oriansj>muurkha: that is done in hex2 and M1 & wouldn't help there <muurkha>gawk's r_get_lhs does something like that, though not in a hash table <muurkha>in Emacs's acl_nfs4_nontrivial there's an interesting case where & is used just to shorten the code a lot <muurkha> nfs4_ace_int_t *ace = &a->aclEntry[i]; <muurkha>this is followed by a 9-line-long if condition which references ace 7 times <muurkha> if (!((ace->flags & ACE4_ID_SPECIAL) != 0 <muurkha> && (ace->aceWho.special_whoid == ACE4_WHO_OWNER <muurkha>of course if the nfs4_ace_int_t were separately heap-allocated, as it would be in Scheme, instead of embedded in an array, you wouldn't need & <muurkha>and you could also say nfs4_ace_int_t ace = a->aclEntry[i]; and then use ace. instead of ace-> <oriansj>and without &, with a bit of cleverness you can make C support garbage collection. <muurkha>but both of those options are less efficient <muurkha>people make C support garbage collection even with & though <muurkha>if you just remove & from C without adding var parameters, you add significant inconvenience and error-proneness in cases like these, and you lose the ability to do multiple return values without declaring a struct type for them <oriansj>like go and implicit struct return type is possible <muurkha>note btw that Golang supports garbage collection (precise GC, even) without removing & <muurkha>because you can't convert pointers back and forth to integers in Golang, GC can be precise rather than conservative; the fact that sometimes pointers go into the middle of an object instead of its beginning does complicate the GC a bit but doesn't make it impossible <oriansj>muurkha: the point isn't that removing features could result in some cases getting worse (which I certainly grant you) but the lack of features such as & could have allowed more time for better concepts to develop and have enough demand to become part of the C language <muurkha>dig.c in nethack does a somewhat similar thing to the Emacs example in mkcavepos <muurkha>and then it uses lev-> extensively, both for reads and writes <muurkha>it never changes x and y, so you could certainly have rewritten things like <muurkha>and that wouldn't even necessarily be less efficient, although making it efficient does demand a little more from the compiler <muurkha>but it would definitely be more error-prone <muurkha>I think that if your "memory model" (in the sense I use the phrase in http://canonical.org/~kragen/memory-models/) supports nested objects, you're going to experience a lot of pressure toward adding &. even if you already have var parameters; the Pascal that the original Macintosh System apps were written in had var parameters, and they added & (spelled @) <muurkha>(I think actually they got it from some Pascal compiler on the Apple ][) <muurkha>though maybe they mostly used that for interfacing with routines written in assembly! <muurkha> SetRect(mainBits.bounds,80,120,80+416,120+240); <muurkha>which is, interestingly enough, only about 3500 lines of Pascal and 2000 lines of assembly <muurkha>as I see it, the main difficulty with & is only a problem for machines with addressing of 16 bits or less: it means that all your data has to live in the same address space. in particular, you can't store all local variables outside of what wasm calls "linear memory"; your subroutine parameters, or at least some of them, have to live in the same address space as your heap and globals, so that you <oriansj>muurkha: I grant you all of that but in all of those cases & still probably wasn't the right idea. <muurkha>what do you think would be a better way to handle them? <oriansj>well my first question would be why not lev = levl[x][y]; <oriansj>unless the point is to just update levl[x][y] with a value; then why bother with the extra pointer <oriansj>struct foo { struct bar[x]}; struct baz { struct foo [y]}; then it becomes baz->foo[y]->bar[x]->seenv; <muurkha>if you do lev = levl[x][y]; then the later lev.seenv = 0; won't have the desired effect; it will update your local variable rather than the cell of the level <muurkha>previously it says things like if(IS_ROCK(lev->typ)) return; <muurkha>which would work fine with a copy of the cell <muurkha>something like a C++ "reference" would work fine for cases like that, though, and that doesn't pose any extra difficulty over Pascal-style var parameters <muurkha>in fact I think the reason Bjarne added references to C++ was to get Pascal-style var parameters <muurkha>(or maybe so that operator[] could return something you could not just read but also assign to? which is kind of the deal with the hash table or gawk use case for &) <muurkha>(although in fact operator[] doing that in C++ turned out to be a terrible idea because it results in autovivification) <oriansj>only if the language doesn't do strict array size checks <oriansj>and require the programmer put checks around dynamically sized bits <muurkha>hmm? what do array size checks have to do with anything? <muurkha>*returning* references like operator[] is definitely a potentially unsafe thing to do, btw <muurkha>in order to have something to return a reference to, it allocates a node, inserts it into the red-black tree, and returns a reference to one of its fields <muurkha>you define your operator[] to return a proxy class instead of a reference, and then you can detect whether it's being written or read <oriansj>and here I was thinking if you know m[1] is beyond the length of m would just be a compile error; <muurkha>std::map doesn't have a fixed length; you can insert into it at runtime <muurkha>oh, well, as you know, it's often useful to program with flexible containers that expand at runtime to accommodate whatever you insert into them :) <oriansj>as the second you do something like m[i] where i exceeds the length of your array; you then have to do either a new array and copy (reference problems all around) or hope you can can extend your array (or throw an exception on potentially a boatload of operations) <muurkha>but m isn't an array, it doesn't have any arrays in it anywhere <muurkha>it's a std::map, which is implemented as a red-black tree <muurkha>but inserting into a std::map can indeed throw an exception! <oriansj>m[i] only can be fast if it is an array (or vector which is just another word for array with a few extra bits) <muurkha>and std::vector also has the other problem, in which push_back invalidates all previous references and iterators into the vector <oriansj>and in the case of m[i] going off the end, why not just return a NULL pointer <muurkha>what would it mean to not have m[i]? <muurkha>what would it mean to go off the end of a std::map? <oriansj>well as std:map is just a red/black tree so why use it like an array/vector when it can be used like a tree. <oriansj>to go off the end of a std::map would be to ask for the subtree of a tree that doesn't exist <doras>stikonas: regarding dynamic linking with musl, there's the matter of either using the `musl-gcc` wrapper when building instead of `gcc`, or patching GCC itself to natively build against musl's linker. Do we have a preference? <stikonas>I would say patching gcc makes more sense <doras>Because it gets a bit annoying with C++, where musl doesn't provide a wrapper script for the `g++` command. It can technically work with some manual changes based on my tests, however. <stikonas>musl-gcc wrapper probably makes more sense if base system is glibc <stikonas>doras: does newer GCC include this patch? <stikonas>I looked at parent dir and there are no patches for newer gccs <doras>So far I only tried the wrapper scripts, but with some modifications I got it working for C and C++. <doras>It does requiring specifically setting environment variables during configure, however. e.g.,`CC=musl-gcc`. <doras>I don't mind trying the patch to see what it does. <doras>It doesn't have a clear license though. <stikonas>though I don't know if that assumption is good enough <stikonas>I mean the author of the patch either has to agree that is is GPL or they are infringing GPL <stikonas>though if we have support for dynamic linking in live-bootstrap (which is good to have as some stuff depends on it) it raises a question how much we want to use it <stikonas>I think fossy would prefer to only use it when strictly required <AwesomeAdam54321>Why would static linking be better than dynamic linking for the bootstrap? <stikonas>AwesomeAdam54321: better is probably not the right description, more convenient for some things <stikonas>e.g. you can create a self-contained binary package of some application (say bison or bash) that you can reuse anywhere <stikonas>with dynamic linking you need to think about dependencies <stikonas>well, there is some size overhead but the whole live-bootstrap package repo is only about 137 MiB (not counting linux kernel) <stikonas>though perhaps at some point we should start thinking about dependencies between packages <oriansj>AwesomeAdam54321: well static vs dynamic linking; is just an engineering tradeoff between trying to avoid dependency hell and not wanting to do a recompile everything everytime a library is updated. <oriansj>although functional package managers (like nix and guix) tend to make the second point invalid and static linking just becomes a better idea. <stikonas>well, live-bootstrap does not use that many libraries yet, so we mostly used static linking <stikonas>C library is the main thing we use, but then we switch a lot between incompatible libraries <stikonas>e.g. once you upgrade from musl 1.1 to musl 1.2 you need to rebuild anyway ***lukedashjr is now known as luke-jr
<ericonr>stikonas: it very much is ABI compatible <ericonr>you do want to recompile the world on 32-bit architectures, though, because otherwise any ABI boundary that uses time_t will be inconsistent across applications built with the older or newer headers <stikonas>I thought time was 32-bit in musl 1.1.x but 64-bit <doras>stikonas: I got it working by backporting a few upstream commits instead. I'll upload a PR soon. <stikonas>and libtool PR is almost done from what I can see <doras>Apparently I got disconnected again and probably a few messages of mine didn't reach IRC. <doras>^ dynamic linking with musl in live-bootstrap <doras>Most of Python 3's in-tree modules are building and running successfully on the live-bootstrap side now. <doras>For some reason zlib doesn't, which fails Python's own `install` stage. I'll need to take a look why.