<stikonas>rickmasters: just a small clarification. Your readme says "The internal shell will also execute any file that has previously been written (by hex0). " Should this be "The internal shell can also execute any file that has previously been written (by hex0)." ? <stikonas>since that helloworld.src has to hex0 commands at the end but then follow by "./bootstrap-seeds/POSIX/x86/kaem-optional-seed ./kaem.x86" <stikonas>from what I've been looking your kernel integrates both hex0 and kind of shell/kaem functionality as built-in <rickmasters>stikonas: yes, my shell sucks so highly recommend runnning your commands from kaem <stikonas>well, yes, I was just thinking whether we can use it to build kaem seed <rickmasters>stikonas: not sure what you mean, it uses hex0 to build kaem <stikonas>hmm, let me check that helloworld.src first <stikonas>maybe I was saying something that makes no sense <rickmasters>right, thats a built in hex0 that builds kaem-optional-seed <rickmasters>with some work, the internal shell could run hex0-seed to build kaem-optional-seed but not sure it matters, the internalshell only parses one arg, hex0-seed needs 2 <rickmasters>i mean, internalshell only parses one arg for non-built in commands, the build-int hex0 and src parse their own args <rickmasters>yeah, kaem is better all around, best to just pass off all further execs to kaem <rickmasters>for example - internalshell only looks at the first character - h or s and assumes hex0 or src, so good think kaem starts with a k :p <rickmasters>worth noting there is no way to create binary files without going through hex0, no way to cheat :) <stikonas>yeah, we do need hex0, no way around that <stikonas>I did have minimized "micro-kaem" for riscv64 with hardcoded commands as a proof of concept but we don't really use it <stikonas>anyway, it builder-hex0 looks really nice, I was just asking some questions to speed up my understanding <rickmasters>btw, the reason i call it a "builder" rather than a kernel is that it is arguably a shell on a unikernel, oriented towards compiler <stikonas>well, naming is one of the three biggest problems in computer science <rickmasters>it started just like the "mini" - it was only a compiler, but it evolved to being more kernel, in retrospect, it should be kernel-hex0 or something like that <rickmasters>or something silly and memorable, like i dunno, "chongo" <stikonas>haven't seen this... Probably a bit too young <stikonas>ok, one thing we can't do in builder-hex0 is load binary files... <stikonas>it expects nice text files with newlines <stikonas>(I was trying to cheat a bit and upload mes-0.24.tar.gz rather than mes source files) <stikonas>if that makes hex0 kernel smaller then it's the right thing to do <stikonas>it just means we have to use M2-Planet and not mes for the next kernel <rickmasters>but only allowing text files was my way of ensuring every binary was properly bootstrapped / hermetically <rickmasters>but if the file was previously created by the bootstrap, then it seems ok, so a multi-phase approach should allow carrying over binaries <rickmasters>i.e. write binaries to a file system, lay down a new kernel and reboot seems fine to me <rickmasters>its a subtle question about what fits the principles of bootstrapping <rickmasters>loading tar.gz directly could be ok if you can verify they are (via sha256) previously created by the bootstrap <stikonas>but yes, we can do something like base64 encode binary in stage0-posix <stikonas>stage0-posix build untar, ungz and sha256sum for checksumming further sources <rickmasters>stikonas: when you read this tomorrow - i think a "bin" command similar to src would be reasonable for tar.gz sources, guess we wouldn't need src then? <stikonas[m]>But maybe some binary to ascii encoder would be simpler? <oriansj>and extern which are meaningless in M2-Planet <oriansj>we also support struct foo { union { int bar[8]; char baz[64];}; }; types if one needs them <oriansj>it supports any function with arbitrary numbers of arguments (but there isn't a GCC compatible C function pointer that I know of yet) <oriansj>unmatched-paren: you can use any feature of the C language you desire in M2libc IFF that functionality exists in or is added to M2-Planet <Hagfish>how do people write this stuff? "Consider the difference between a machine “proving” something is mathematically correct, and a human recognising that it’s true. The machine can churn out equations all it likes, but it can’t actually prove that something’s true unless there’s a conscious being, capable of interpreting the data, to whom it’s doing the proving." <Hagfish>basically "a proof written by a computer is only true if a human brain believes it" <Hagfish>that's turning the idea of a proof inside out <muurkha>no, it's not saying the proof isn't true if nobody reads it <muurkha>it's saying it doesn't have a truth value at all <Hagfish>a proof might not be _useful_ if a human doesn't read it, i'll grant them that <Hagfish>but it seems to be arguing on the level of "if a tree falls in a forest" <muurkha>kind of, but it's a bit more defensible than Berkeleyian idealism <muurkha>but I just realized I'm debating the nature of truth again and I promised not to do that <muurkha>so I'll just say I understand both your point of view and the point of view of the person who wrote that <Hagfish>fortunately the article wasn't about the nature of truth, but about whether AI is achievable <Hagfish>For instance, if scientists are correct that human language can only roughly approximate what’s actually going on at the level of quantum physics, then a simplified linguistic code that attempted to replicate the quantum processes going on inside our brains would always fall short. But we know of no way of “coding” that doesn’t rely on language — and we’re forced, therefore, to use one very small part of our own intelligence <Hagfish>they're saying that we can't put quantum physics into human language, therefore we can't simulate quantum systems, therefore we can't make AI <oriansj>Hagfish: the term AI is meaningless so the question of if it is achievable is therefore pointless <Hagfish>yeah, i think their whole article relies on the lack of definition for all of these terms <muurkha>I'll permit myself one more remark: Hagfish, can submarines swim? <oriansj>much like the definition of what it means to be human is poorly defined (as is quite well covered in the comic book series the Incredible X-Men in the Sentinals) <Hagfish>yeah, the problem is pinning down the definitions (and defining the right things) <Hagfish>"Abductive reasoning is the odd one out: defined variously as guesswork, speculation and intuition, it is the much more mysterious, but ultimately more important, form of inference that guides our everyday “common sense”." <Hagfish>so they avoid the issue of defining "intelligence" by talking about "abductive reasoning" which they define as "something like intuition" <Hagfish>which doesn't really illuminate the problem <oriansj> So my view is I don't have much to add to questions that aren't defined nor am I able to solve. <Hagfish>i think the author is deliberately picking nebulous concepts so that they can't be challenged on them <Hagfish>if they said "a machine will never be able to solve problems of this form...." then that would be a testable hypothesis <oriansj>Hagfish: Any claim made without proof can be dismissed without proof <Hagfish>is there a name for that principle, or can i call it Orian's Razor? :) <oriansj>and non-falsifiable statements are as productive as a Burly Biker dude telling me they give the best blowjobs; 1) it doesn't matter and 2) I could have gone my entire life without knowing that 3) learning it changed nothing. <oriansj>Hagfish: I believe it is called Hitchens's razor <oriansj>and the second is a variation on: Russell's teapot <Hagfish>i wouldn't say that learning it changed nothing, i'd say that (if it's true) it actually leaves you in a worse position than not knowing it <Hagfish>i think in the HHGTTG film, a character asked something like "would you rather be right but sad, or wrong and happy?" <oriansj>Hagfish: not really; the worst that happens is my friend Jack breaks up with his boyfriend again and I give Jack the dude's number. Maybe that'll finally give him a relationship that lasts more than 6 weeks. <Hagfish>yeah, the value of true knowledge extends beyond our own happiness (although ideally helping others would make us happy) <oriansj>Hagfish: Happiness is a cheap trick used to keep humans alive and addicted to costly behaviors while starving in the wilderness. Responsiblity is what tends to produce the best emotional outcomes in the long term. <Hagfish>by "costly" do you include "long payoff"? <Hagfish>but yeah, happiness is a mechanism/heuristic that can be tricked into chasing unproductive goals <oriansj>much like how the repeated use of Anti-Inflammatory medication tends to result in Chronic pain conditions. The use of the application of artifical happiness tends to result in Chronic depression. <oriansj>and the worst is when people start thinking of others in terms of "emotional labor" and "return on investment" <oriansj>Those who give without expecting anything in return (in a healthy community) tend to recieve far more than they give (See collective benefits in the FSF community or even here) <Hagfish>yeah, the free software community is a great example of "it's a nice idea in practice, but it would never work in theory" <oriansj>Hagfish: Ironically I heard the exact reverse about bootstrapping when I started stage0. <Hagfish>yeah, this is a "love to see the doubters all proven wrong" project for me <oriansj>Hagfish: It is a love letter to the world for me. <oriansj>I expected it to take me a couple decades to get as far as we collectively did. <oriansj>But instead I found I wasn't alone in this dream of bootstrapping the world and there was all this passion and joy and fun. The people here are amazing, talented and I am honored for the chance to give them hope and success. <Hagfish>yeah, this project is an existence proof of the unreasonable effectiveness of passionate, creative, and talented people working with a shared vision that benefits the world <Hagfish>what do you think the first hardware will be that actually does the "full" bootstrap? <Hagfish>i.e. hand-entered seed, up to modern linux (possibly by shifting to different machines at various steps) <Hagfish>i've just been looking up about punch card readers, and i think this isn't cheating: <oriansj>well one doesn't need a full CPU and software to do a punch card reader <oriansj>and also one has to be precise by "full" bootstrap; as one must ask do you mean with or without a BIOS/Firmware or do you mean with or without microcode? <Hagfish>yeah, the problem of definitions again <Hagfish>i think i accept that machines have BIOS chips and they might as well be used (until we get really purist about the free hardware problem) <Hagfish>i'm thinking something like a 386 would be a reasonable first platform to target <Hagfish>but i don't know what's a reasonable way to interface that with punch cards (or how else to load the seed binaries) <oriansj>Hagfish: well I believe we should go to the level where people are satisfied. If that means a i386 and a 20GB hard drive; sure why not. But I am game to go to individual transistors and core memory. <oriansj>Hagfish: well a serial port could be used to populate memory with the seed <Hagfish>yeah, i think there will be a community that won't give up until at least it gets down to that level <Hagfish>are there bioses that boot from serial? <oriansj>and I'll be there helping every step of the way. <oriansj>Hagfish: I am uncertain but possibly as there are bios that boot from ethernet (PXE boot) <Hagfish>i'm also not what a punch card reader would have to look like to be able to talk serial like that <Hagfish>i was thinking maybe a punch card to sd card reader, then an sd card to ide convertor <Hagfish>very hacky, but i guess it could be reasoned about <oriansj>well a clock, a driving transistor and a cascade logic for bit selection probably <Hagfish>nice. i'm also not sure what the timing requirements of serial are <Hagfish>like if someone is manually swapping the paper cards over during the boot process :) <oriansj>Hagfish: well a stepper motor probably could do the feeding job (although paper tape would be simpler mechanically to do) <Hagfish>yeah, at that level it becomes more about mechanical and electrical engineering than it does about software engineering <oriansj>indeed and we will probably need to fund that work <Hagfish>there are hobbyists who have made more complicated hardware, of course <Hagfish>but yeah, it's a well-defined project which someone could focus on and implement quickly if they had funding <oriansj>and I have all of my crypto ear-marked for bootstrapping projects <oriansj>and look forward to when those people show up <Hagfish>i wish i could say that investing in a free hardware startup would be stonks, but being involved in the design of core bootstrapping devices is a privilege money can't buy <Hagfish>i'd love it if one day, every new SWE hire at google had to "go to the oriansj-machine" and type in the bootstrap seeds before being issued their first machine <Hagfish>just the act of going through the bootstrap, is a great way of reminding people where technology (and trust) comes from <oriansj>well Hagfish believe it or not, that is actually a revolutionary perspective that most people want to hide for control reasons. <Hagfish>yeah, it is weird to talk about "rituals", but rituals are all about grounding a community in a set of shared experiences and values <Hagfish>job interviews and performance reviews are all types of ritual too, which imply their own set of values <oriansj>The whole "we have magic technology" sort of don't look too close model is really useful for making a great deal of money. So one might argue what we are doing is extremely subversive to modern society. <Hagfish>yeah, it's sort of a sci-fi trope that in the future there is a class of people who don't know and don't care about where their magical technology comes from <Hagfish>but you sort of have to suspend disbelief, because that society couldn't last long before the power structures inherent to that design ripped the society apart <oriansj>Hagfish: *tries really hard to not point at the state of technology already around most of the people all of the time* <Hagfish>yeah, the problem is just getting worse, but bootstrapping at least ensures there is a golden thread stopping us from all floating away into the void <oriansj>Hagfish: we are well into the void; Just look how hard it is becoming to even buy libreboot hardware these days <Hagfish>the hardware situation is really worrying to me, especially the societal implications of assumed TPM availability <Hagfish>accidentally unauditable hardware i can almost excuse, but deliberately unauditable hardware i treat as an attack <oriansj>well we do have lights down that tunnel; iCE40 FPGAs have Free as Freedom programming stacks available and more FPGAs are gaining that every month <oriansj>Libresilicon is getting started and in a couple years might finally be good enough for us to start supplying our own standard designs. <Hagfish>that's a nice lifeboat or island that can sustain things for a while, at least <oriansj>With cooperation and funding, we could leverage late generation Commercial fabs to get custom ASICs out now <oriansj>a 90nm chip might not be cutting edge but it'll be good enough to do modern software development <Hagfish>wow. i do have some hope that modern hardware progress might stall, in the sense that everyone's needs are already fulfilled <Hagfish>we don't need phone screens to be either twice as big or twice as small <Hagfish>perhaps there will be a trend towards AI running locally instead of in the cloud, which will probably be a positive for privacy <oriansj>Hagfish: it already stalled in many ways. <oriansj>One could even say my job's machine (which is a modern beast running shitty software because Governement) is an order of magnitude slower than my librebooted x200 <Hagfish>i wouldn't like to say whether hardware stopped improving because people didn't want to buy more powerful machines, or people stopped buying more powerful machines because the hardware stopped improving <oriansj>ask yourself this how many years has it been where the average RAM was 16GB? <Hagfish>i've stopped following specs, and stopped upgrading my own hardware <Hagfish>i wonder how many apps the average office employee runs <oriansj>It has been a decade of effectively zero progress <oriansj>Hagfish: I have no clue but knowing a few drones: Outlook, Excel, Word and PowerPoint is probably the full limit of it (not counting "Web Apps") <Hagfish>if you gave them a computer from 10 years ago, and let them not have to run Electron apps, they might think the PC is faster <Hagfish>yeah, and even though Word is probably still sluggish, it's not likely to get better by throwing CPU and RAM at it <pabs3>MS Office is a web app for most people now isn't it? <oriansj>Copying a file at 32KB/second is somehow fast on that crap and I remember a PC clone in 1991 which was faster than that shit. <oriansj>I literally had the following experience: Copying files to a network share. It was 300x faster to boot up a VM, mount the share in Linux and use Rsync to copy the files than to do copy and past in Windows <oriansj>^past^paste^ (or even robocopy or Teracopy) <oriansj>Or that "pause" that occurs when you are typing and you are just sitting there and waiting for the computer to catch up and you wonder how long God has abandoned us all. Before you wonder if emacs could be used to edit word docx files because you swear an emacs macro could eliminate half that crap <oriansj>oh and a bunch of Excel file and email file generators because What government workers don't love to create a bunch of excel files (manually) when automation can do that. Or a bunch of emails (manually) when the data could be generated automatically <Hagfish>i think that "no code" services might be able to make in-roads here <oriansj>I'm fighting an up hill battle to getting them to run powershell scripts <Hagfish>they just need to take your script, build a pretty UI, charge some insane multiple of how much it all cost, and then jump through the government procurement hoops <Hagfish>governments want systems to be "legible" to themselves, which means lots of written contracts, and not pockets of technical expertise <Hagfish>that's also how they tend to treat citizens <oriansj>I mean I literally have given code to companies and requested that they submit a bid which would deliver the functionality provided by that code I gave them and "support it" And those companies are happy with their multimillion dollar/year support contracts and the State of Michigan is happy that things are working and the citizens are rioting *again* <Hagfish>it does seem like there is some arbitrage that could be done here that even an administrator/bean counter could be happy with <Hagfish>i was going to say that you should get a cut of any support contract, but there are probably conflict of interest rules that you'd need to not break <oriansj>It'll replace a $8M/year solution; a $6M/year bid would probably win if one could promise a 3 month delivery. <Hagfish>"if 19 > Applicant.Age >= 16 and Applicant.FPL > 109:" <oriansj>well yes there are lots of laws about conflict of interest and honestly I've never taken a dime. <Hagfish>wow, you're literally solving government with software <oriansj>netted the State of Michigan $30M in savings in the last 6 years working for them. <Hagfish>they should be paying you that much just because of the national security impact of your boostrapping work <oriansj>my job title is literally "Computer Security and Reliability Engineer" and it took them 5 years to promote me from an ITP-11 to an ITP-12 <oriansj>and it was my generating of Excel files to send to IRS auditors which is why the promoted me. <Hagfish>like using a screwdriver to remove a stone in your shoe. at least they appreciate some of your usefulness <oriansj>the average promotion path from an ITP-11 to a ITP-12 is 2 years; but because I provided documentation and evidence that Windows 10 usage in several departments would violate federal law; I got stuck in the "naughty box" <Hagfish>"if we promote him any more, we'd have to tell him about all the cool alien technology we've recovered, and we can't trust him with that" <oriansj>Not at the state government level; it is more a game of herding cracked out goats <oriansj>They don't give a flying fuck about anything if there aren't riots, a judge demanding they do something or the Feds will stop giving them piles of cash. <Hagfish>there does seem to be some law of organisations/projects that if you cut the resources below a certain level, the project ends up (ironically) becoming exponentially more expensive <Hagfish>just like the observation that "it's expensive to be poor" <Hagfish>and if you rush a project, it ends up taking longer than if you had taken your time ("slow is smooth, smooth is fast") <oriansj>and if your hiring funnel for college graduates (ITP-8 positions) pays less than McDonald's well you have one hell of a problem. <muurkha>oriansj: do you know about dollar auctions and grad student stipends? <oriansj>and yes that ITP-8 position requires a 4 year degree in IT and 2 years work experience to offer a job paying $11/hour <muurkha>I think the ITP-8 position is a dollar auction, similar to grad-student positions <muurkha>what percentage of ITP-8s get promoted rather than leaving? <oriansj>muurkha: Entry level programmer positions in Lansing pay $108K/year; The State of Michigan is offering $22.9K <muurkha>right. so what percentage of ITP-8s that get hired will stay? <oriansj>about 80% stay but only 40% of those will get promoted <muurkha>interesting, so it's not a dollar auction kind of thing I guess <muurkha>the payoff I assume is bulletproof job security <muurkha>the opportunity to sacrifice your own well-being in the interest of public service, perhaps? <oriansj>Government hires those with paperwork who generally can't go else where for more money <muurkha>great salaries if you can get promoted to ITP-14? <oriansj>ITP-11 pays $25/hour ITP-12 pays $42/hour and ITP-14 pays $60/hour <oriansj>well the reason most can't go elsewhere is because they can't program <oriansj>yet somehow that doesn't prevent them from being promoted. <oriansj>I mean literally even if you let them pick the programming language they can't program. <oriansj>You can see it in the commit history <muurkha>so in theory if you can get enough seniority you can make US$120k a year for sitting at a desk and ineptly but obediently programming <muurkha>sounds better than when I was bottling bubblebath in a bubblebath factory <muurkha>I remember one of my coworkers gave me a tip on how I could make more money <oriansj>muurkha: you don't need to even be programming <muurkha>he said in the summers you can make more than US$100 a week by mowing people's lawns <muurkha>this was in 01996 so that's probably more like US$230 a week now <oriansj>yeah, it is very low stress environment if you don't care if things don't get done. <muurkha>yeah, if you just do what you're told, you can live pretty good on the taxpayer dole <oriansj>and you get time and a half for every minute over 40 hours <muurkha>but to get there you have to eat ramen through all the years of making US$22.9k <muurkha>which sets up an obedience competition between the ITP-8s <oriansj>or get hired into a a higher level by negotiating well <muurkha>too bad it's not a competence competition <muurkha>you can negotiate? I thought government grades were pretty much always based on objective criteria <oriansj>I got hired straight in as an ITP-11 <oriansj>making half-way up the ITP-11 payscale <muurkha>not great but better than US$22.9k. and rents are lower in Michigan <muurkha>my landlord wants to double my rent from US$200 to US$400, including the utilities <oriansj>I've got a paid off house (4 year mortgage) and live ok with my wife and kid <muurkha>I have a 40m² efficiency on the fourth floor facing a major avenue <muurkha>I think I can probably get a better apartment for less money someplace nearby <oriansj>I'm 2 blocks from a bus stop and 2.5 blocks from a Kroger <muurkha>yeah, right now I think I value quiet, outdoor space, and bedroom doors that close <muurkha>my current apartment is 0.2 blocks from a bus stop, 0.8 blocks from a supermarket, 1.1 blocks from another one, 2 blocks from a third, across the street from a halal butcher shop and a health food store and a greengrocer and another butcher shop <oriansj>well Michigan streets are very driving centered <muurkha>and 0.8 blocks away from a rotisserie Chinese food place that sells you whatever mix you want of all their salad and hot food for about US$8 per kilo <muurkha>the nearest grocery store with Kroger-like selection is about 8 blocks away though I think <muurkha>but the problem is that every time a bus goes by we have to pause the conversation in the apartment <muurkha>and the nightclub next door made it necessary to wear earplugs to sleep during the weekends <muurkha>some of the neighbors reported that the vibrations cracked their plaster <muurkha>they seem to have been shut down now, which is pretty unfair since they were there before the apartment building <oriansj>yeah, the loudest thing around here usually is just the rain <muurkha>and, being an efficiency, there's no privacy except when you're in the bathroom <oriansj>well come to Michigan, I'll be a solid reference <muurkha>right now I need to get my Argentine immigration stuff straightened out <muurkha>which involves a bunch of paperwork, getting married officially, that kind of thing <muurkha>I've convinced my girlfriend that visiting the US will be great, but she definitely doesn't want to move there <muurkha>her parents and her grandmother are old; if we were to move there, there's an excellent chance she would never see them again <muurkha>I visited Ann Arbor a few times when I lived in Dayton <muurkha>yeah, I was too arrogant and insecure to really take advantage of the great opportunity that was <muurkha>too tied to proving myself smarter, but not disciplined enough to get impressive things done. really pretty similar to today, but worse <oriansj>I think we all go through that when young. <oriansj>and how many experiences we get learning from those better than us <muurkha>hmm, maybe? you'd think having some great mentors would make me humble, but then again you'd think taking 6 hours to debug a bug that seems trivially obvious in retrospect would also make me humble <oriansj>muurkha: humble people who stay in their place, never change the world. Stay awesome <muurkha>I think I'd probably change the world more for the better if I were better at remembering my fallibility and not hurting people's feelings <oriansj>muurkha: depends on the ways you wish to change the world. <oriansj>if you wish to impose your will upon others, money is the best tool to use. <oriansj>But if you wish to make the world better for others, sometimes being the helpful asshole is something you need to play. <oriansj>yes there are ways to call people on their bullshit without making them feel attacked but at the end of the day bullshit needs to be called out and replaced by something that actually works. <oriansj>and sometimes the cow does have to be taken out and shot; before you can clear out their pile of shit <muurkha>well, I can be wrong about which things are bullshit <muurkha>in terms of money, I've done a pretty piss-poor job of that <muurkha>moving to Argentina in 02006 has been very educational but disastrous for my earning capacity <oriansj>muurkha: well present evidence which would support why you believe the things are bullshit and if you are wrong, get evidence from the other side and expand what you know about the world <muurkha>yeah, if they care about presenting it <muurkha>but in some cases they'll just dismiss me and give me no further information <oriansj>muurkha: some people just suck and avoiding them is sometimes the best route <muurkha>I think my ideal situation moneywise would be spending 1-3 years in the US working on a startup, as an employee, starting maybe next year <oriansj>well there is an ITP-14 architect position open in my department you would be great in assuming you are willing to go in once a month <muurkha>"on a startup" so there's no expectation that I'll spend the rest of my career there <muurkha>I don't think I can get hired for government jobs, I don't have so much as a high-school diploma <oriansj>look we are losing the Building and being doing remote work the last 2 years with only security team staff going in once a month <muurkha>I got admitted to university at the beginning of my senior year of high school, then dropped out after a year <muurkha>reasonable but not great grades, but I wanted to do something "real" <muurkha>so I went to work at Taco Bell in South Dakota <oriansj>to be honest, I only went to college after that whole dumpster fire where they stole my retirement and my resume didn't seem real enough for anyone to hire me outside of a Computer Lab <muurkha>oh right, I forgot about that nightmare <oriansj>So took classes for 2 years and got my 4 year degree; learned absolutely nothing but it filled a checkbox <muurkha>so yeah, about 9 months later I got my first sysadmin job <oriansj>My job would definitely fall in the duct tapers and box tickers categories <muurkha>but it pays the bills while you're doing stuff that really matters <oriansj>Security is all box ticking and reliability engineering is just Duct Taping shit together good enough that people get paid and there are not riots in the streets again. <muurkha>you can't stop the riots, they'll be worse the next time <oriansj>Currently I am enabling children to get childcare and using C# to generate notice emails for people who failed to put the correct information in Jira <oriansj>but I don't get Database access to Jira and the API limits the results to 50 per query and 1000+ are generated in a day <muurkha>sounds like annoying problems to solve <oriansj>and the tools doing the work can't be curl, jq, sqlite or anything useful; so I have to write them in C# to work around that crap. <muurkha>can you use free-software libraries in C# or do you have to duplicate them yourself? <oriansj>I only got 2 free-software libraries approved System.Data.SQLite and EPPlus -Version 4.5.3.3 (because of license change) <oriansj>but I know how to work with bear skins and stone knives to get serious work done <muurkha>just, in the sense that your job is hard for a lot of reasons that have nothing to do with the actual problem of getting childcare to children <muurkha>and everything to do with irrational and contradictory policies that create needless inefficiency <oriansj>and for extra fun I am not given permission to run the tools that generate the reports but instead I have to train a person who isn't a programmer and never used a shell in their life to enable scripts in powershell and run the commands needed to generate everything. <oriansj>from a dataset in Jira no one actually knows <oriansj>to generate alerts based on things that haven't even been decided on yet <oriansj>but needs to be done last month before I even started. <oriansj>so relative to that, bootstrapping work even writing hex0 programs is relaxing and I am glad my wife considers it my decompression hobby. <muurkha>well, the problems are actually within your control <muurkha>the job sounds like those experiments where they would give puppies electric shocks when they tried to cross the floor to get to the food <achaninja>oriansj: You seem talented enough to change if its unpleasant <muurkha>changing a government agency is hard <Franciman>thanks oriansj for the list of features of m2planet <doras>stikonas: so I think xbps is a separate issue. xbps-create(1) says "The xbps-create utility creates XBPS binary packages from files stored in a directory. **The files must have correct permissions and location within this directory.**" <stikonas>hmm, fossy might be more familiar with xbps but I thought it's basically just tar with extra metadata <doras>I noticed that we either create a tarball or an xbps package, and it seems that the xbps packaging command doesn't offer a similar "--mode" to what tar provides. <stikonas>and there is no way to change mode before that? <stikonas>i.e. before tar'ing or xbps while we are in destdir <doras>We could probably chmod, but it's going to need a per-file treatment. <doras>It seems that are packing strategy doesn't involve manually listing all the files to be packaged, like RPM .spec files do. So we don't have some finite list of files in each package where we can specify the exact permissions we want for each file. <doras>Unless we transition to this packaging method, it's not going to be maintainable to chmod specific files. <doras>I think our best option at the moment to get reproducibility around file permissions in packages is to implement the umask stub in the mes libc so we can set it as early as possible in the bootstrap. <oriansj>muurkha: well it is more of fear of blame and responsibility than anything actually evil. So a good deal of the stress and pain is just me forcing progress into an environment where failure that followed the approved procedure is comfortable. One doesn't save $30M in 6 years by themselves without having do a lot of heavy lift. <oriansj>achaninja: well the big reason it is a huge pain is because I am forcing progress upon them *again* and like getting them to convert from ClearCase to git; it'll be me doing a boatload of work with zero help and a bunch of active resistence until it becomes "too late it is the process now" <doras>What's the point of the "lib/m2" directory in mes? <janneke>it contains alternative implementations for m2 <janneke>code that cannot be (easily) merged with gcc-compatible code <doras>I'm a bit new to all of this. janneke, do you mind providing me some pointers on what steps I need to take to implement the umask function in mes? <doras>It should be a fairly simple function that simply calls the relevant system call as far as I can tell. <doras>I'm not sure if it should be in lib/mes, lib/linux, lib/unix, or all three. <janneke>doras: as it's a linux system call, you would create lib/linux/umask.c, and add it to build-aux/configure-lib.sh <janneke>the stub can stay, for non-linux systems <doras>janneke: thanks. I will try. <janneke>doras: good luck! note that you'll have to update syscalls.h for all supported architectures <stikonas>doras: but xbps only happens once we have full libc, you can set umask then <stikonas>but yes, adding stuff to meslibc is always good too ***lukedashjr is now known as luke-jr
<rickmasters>stikonas and everyone: i am prepared to change builder-hex0's "src" command to load bytes instead of lines of text <rickmasters>FYI, this is a breaking change, but seems like a good improvement <Hagfish>would it make sense to make the command require a hash to run a binary, just to keep people honest? <Hagfish>you could add a --just-trust-me-dude option for people who don't have a hash <Hagfish>(of course having a hash doesn't mean that the binary was properly bootstrapped, or even reproducible at all, but it would set expectations) <rickmasters>HagFish: i think I'm going with "trust the user to do the right thing" <stikonas[m]>Binary vs ascii does not really show what is source and what is blob <rickmasters>yeah, you can already convert anything to hex and load it that way <Hagfish>i was just thinking of that old principle "make it easy to do the right thing, and possible to do anything else" <furrymcgee>why is length required for src? can you read until EOF? <stikonas>furrymcgee: there is no such thing as EOF on disk <stikonas>EOF is something that more powerful kernel with sophisticated file system can implement <rickmasters>furrymcgee: the sources are all appended together in one big shell script <stikonas>with no easy way to tell where file ends <stikonas>I guess that's the simplest file system ever <stikonas>I guess somewhat similar to tar file, just simpler <rickmasters>everything is sharing the same stdin which gets handed off to the built-ins when they are running <stikonas>can't binary files that we upload contain zero too? <rickmasters>yes, because src will absorb zeros, but once its done, it hands stdin back to the internalshell which expects a command or an ending zero <stikonas>furrymcgee: this won't work due to the thing discussed above <muurkha>furrymcgee: traditionally eof is outside the range of possible bytes <muurkha>yeah, it's often -1 even on systems where char is signed char <furrymcgee>yes, either line based containing no zero, or binary with zero. <muurkha>stikonas: I was thinking that with the BIOS system calls the simplest file system ever is bootOS's: every file is 512 bytes <muurkha>but if you allow a few more bytes of code in the filesystem, you can revise that to "every file is one track in size" <rickmasters>at the syscall interface, there is a separate return value indicating number of bytes read - zero is eof (or an error) <muurkha>in standard POSIX an error is negative, but that's probably a distinction without a difference in this case <doras>stikonas: so when do we first have umask? After musl is built? <stikonas>umask built-in not until we rebuild bash <doras>We need to execute the new bash too, not only rebuild it, right? <stikonas>bash is always executed afer it is rebuilt <doras>So the first point we can use umask is in sysc/run2.sh, I assume? <stikonas>right now yes, but that might change to the first bash in sysa if gash is ready <doras>And when do we start creating xbps packages? <stikonas>then the order will be gash->...->musl->heirloom-tools->bash <stikonas>somewhere in the middle of sysc, way later <stikonas>or you can add a step to rebuild bash-2.05b after musl <stikonas>anyway, for xbps packages it's late enough that it should be fine <stikonas>it's the early tar's that might be trickier <doras>I will try using umask at the top of sysc/run2.sh, hopefully it's enough for now. <stikonas>(I'm not sure if I'm doing something wrong or whether we hit kernel limitations) ***ChanServ sets mode: +o janneke
<rickmasters>stikonas: i see "out of memory" errors, the kernel only supplies 1M of stack but would it need that much to display hello <rickmasters>stikonas: its a good start, but man that's a lot of code <stikonas>well, like I said, next kernel can be built with M2-Planet if we can't run mes one builder-hex0 <rickmasters>stikonas: i'm taking a look, how can i get that mes-0.24.tar.gz? <stikonas>rickmasters: I can upload my test commits <stikonas>(and I'm trying to test without nyacc in GUILE_MODULES now) <rickmasters>there is one in live-bootstrap but it appears to be different <stikonas>before mes-0.24 we used some fork with M2-Planet port but now everything is upstream <rickmasters>ok, repro'd the problem, wow it goes crazy with sound effects :p <stikonas>but mes in general spits huge number of never ending errors... <rickmasters>no, just qemu, i think i'm getting bells from the random console output <stikonas>well, there is lots of random console output even on linux kernel if something is not correctly set up <rickmasters>stikonas: have to step out for a bit, but will take a closer look later <doras>stikonas: looks like it was enough. <doras>Now I'm only left with a hash mismatch on util-linux that seemingly ends up with binary content difference around block/disk-related utilities. <doras>I also get this mismatch with the chroot-based bootstrap. <stikonas>might be something to do with your kernel? <stikonas>if you are using similar kernel for qemu <stikonas>if configure picked up some difference... <doras>When I compared it last time, it seemed that configure had an identical output. <stikonas>you might have identical output in both cases due to your kernel... <stikonas>doras: if you can try with the kernel from live-bootstrap <stikonas>(the one that is built and booted into in sysb) <doras>I'm comparing a QEMU bootstrap that gets me the expected hash with a chroot/bwrap bootstrap that give me a different hash. Doesn't the former use sysb's kernel? <doras>It's annoying, the output lines are truncated... <doras>Oh, util-linux is built as part of sysa, so it doesn't use the sysb kernel. <doras>But still, my QEMU bootstrap is running with the defconfig of Linux 5.17.6, so sysa is built on a fairly recent kernel. <doras>Similarly I used 5.17.5 for the chroot/bwrap bootstrap, although with some out-of-tree patches. <doras>I'll try to compare the build directories in the two cases. <doras>The thing is, the very same util-linux version ended up with the expected hash just a few months ago. <doras>It seems that most of the change is due to object files being ordered differently inside the executable. <doras>As if they are linked in a different order for some reason. <stikonas>janneke: is psyntax.pp in mes preexpanded? I seems to have a comment ";;; This file is generated from psyntax.ss." <doras>stikonas: as far as I can tell, most of the change comes from the fact that /usr/musl/libblkid.a's internal objects (.o files) are in a different order. Their sizes are identical, so I'm going to guess that also their content. <doras>I'm not sure what to make of it. <stikonas>fossy might now better about messed up ordering <oriansj>oh god, please tell me janneke isn't doing a repeat of that whole snarfing the C code but on the scheme side this time. <doras>As far as I can tell "ar cru" is used to link libblkid.a, and the order of object files passed to the command in each case seems the same too. (although my output is truncated, so I don't see them all) <doras>stikonas: do you think we could build a newer binutils before util-linux is built? Maybe building it with --enable-deterministic-archives will help. <stikonas>doras: we already build with --enable-deterministic-archives <doras>Hmmm... we seem to have a patch for this. ***ChanServ sets mode: -o janneke
<janneke>but yeah, it's been pre-expanded using guile-1.8 <doras>stikonas: it claims only the object order is different. <doras>I'd rather have "ar" create archives in a consistent order than building util-linux a second time after fixing the order. <oriansj>janneke: so we are going to have to do another psyntax bootstrap? <oriansj>or is it something not needed by mescc at all and could be just removed? <stikonas>doras: well, if you can find an issue, we'll apply the patch <stikonas>yeah, we should either bootstrap it or try to remove it 