IRC channel logs

<anatoly>is it the first case of an arm hardware running mach kernel?

<Gooberpatrol66>anatoly: probably m1 macs beat that record

<Gooberpatrol66>solid_black: you made it into phoronix again https://www.phoronix.com/news/GNU-Hurd-AArch64-GCC-14

<almuhs>I've just compared the rumpkernel's debian patches with the NetBSD 10 source code. I found the patches that needs updating: crossbuild, linux, netbsd-buils.diff, PAE, path_max.diff, ps-comm.diff and rumpuser-evcnt.diff

<almuhs>as curious data, pixide has been removed from NetBSD source code

<almuhs>well, continues, but in ./src/sys/dev/pci/piixide.c

<almuhs>libata has been dissapeared too

<almuhs>the src/sys/rump/dev/lib/libpci_piixide and src/sys/rump/dev/lib/libata has been dissappeared in NetBSD 10 source code

<almuhs>after recheck, i've just notices that libpci_pixide is created in the patch

<almuhs>same with libata

<anatoly>Gooberpatrol66: I meant gnu mach and I bet you knew that ;-)

<Pellescours>sneek: later tell almuhs I tried upgrade src sources on debian package to netbsd10, only netbsd-builds.diff and clean_external needed to be upgraded (lines number changes). It started to compile well unfortunately my VM froze and I did not finished. I think it

<sneek>Okay.

<azert38>solid_black: I was wondering if writing to memory registers need to be done with barriers

<azert38>And that might be the reason the gic and general timer seem to be not configured

<azert38>Gooberpatrol66: I’m always astonished by the amount of latent violence, hatred, frustration, cope and seething emerging in Phoronix articles. The comment section is pure schizo

<azert2>I can try adding barriers, but do you think this has any chance of fixing it?

<Gooberpatrol66>azert38: they seem to treat software as a competition, they're very invested in ranking things and telling everyone which things they think are good and which suck

<Gooberpatrol66>reminds me of the constant brand wars on 4chan

<Gooberpatrol66>they also don't know very much, one of them said desktop microkernel operating systems are impossible, when the amiga...was a desktop microkernel operating system

<azert>Exactly, reminds me of the worse on the internet writing on /g/

<azert>solid_black: I think we need an equivalent to /arch/arm64/include/asm/io.h , where would you put it?

<azert>Gooberpatrol66: the desktop is one of the element in an os where a microkernel in fact can make a positive impact, and where the performances of a perfectionist os like Linux doesn’t buy much to the user.

<azert>also I think that user interfaces will totally change with the advent of artificial intelligence running local

<solid_black>hi!

<solid_black>Gooberpatrol66: "the GCC 14 compiler has been working on compiler toolchain support for GNU Hurd AArch64" funny they should say that, 'cause I could swear it was me working on it and not the compiler :)

<solid_black>"I truly believe the Hurd project would get much more attention and would be quicker ready for use with a rewrite in Rust. I'm not even joking." there we go, we have official the blessing of a phoronix commenter to rewrite the Hurd in Rust :D

<solid_black>azert: re barriers: I don't know, you can certainly try

<solid_black>but first see if the timer/gic get detected and initialized at all

<youpi>solid_black: I've in general gotten used to "I truly believe you should use foo to do your stuff"

<youpi>it's crazy how much that's a very common pattern

<youpi>than people believe you should spend time rewriting your whole code

<youpi>without even looking at it

<youpi>and pointing out _why_ you would benefit

<youpi>as a matter of fact, it would indeed be beneficial to use a language that handles port refs rules for us

<youpi>but commenters don't usually bother to get down to such details :)

<solid_black>I don't think RAII for port rights would even be that big of an advantage

<solid_black>especially given how unusual is the MIG calling convention

<solid_black>how would you even encode "must take ownership of all resources except for the request port if returning 0 or MIG_NO_REPLY, but not otherwise"

<solid_black>Rust async for be very interesting for servers, instead of heavy threading like Hurd servers do now

<solid_black>error handling with ? would be a huge win, if indeed we convince RAII to play along

<youpi>I'm not saying that rust can do what we want

<youpi>just that something could help :)

<solid_black>but the random commenters on Phoronix and Hacker News who know nothing about Hurd (nor Rust...) other than it being a failure certainly don't care for these details

<solid_black>azert: I'm not sure if we even need Linux-like wrappers for memory accesses

<solid_black>but what do I know

<solid_black>if you think we do, aarch64/aarch64/io.h or something like that is where I'd put it

<solid_black>anatoly, Gooberpatrol66: M1 wasn't the first time Mach ran on ARM either

<solid_black>iPhoneOS came out on 2007, and it ran on ARM

<solid_black>but, this might now even be the first time *GNU* Mach runs on ARM

<solid_black>I've done some reasearch before my starting my own port

<solid_black>there's this wiki page https://www.gnu.org/software/hurd/open_issues/arm_port.html

<youpi>:w

<youpi>EWIN

<solid_black>that states one Vasily Sartakov have worked on an ARM port

<solid_black>I went looking for it online and traced it to this 2011 paper https://cyberleninka.ru/article/n/opyt-razrabotki-i-testirovaniya-vstraivaemoy-mikroyadernoy-operatsionnoy-sistemy/viewer

<solid_black>where they claim that they ported GNU Mach (which they call GNU/Mach...) to run on (32-bit) ARM

<solid_black>and they wrote "a basic C library", so I take it they haven't ported glibc

<solid_black>then they say they got an Ethernet driver to run

<solid_black>there's also a presentation/slides version of this, where they mention starting /hurd/pfinet

<solid_black>however I don't buy that, since I don't think the real pfinet would've worked without glibc and a basic Hurd system

<solid_black>there's this comment on Habr from 2013 https://habr.com/ru/articles/171033/#comment_5934099, where he says he "worked on an ARM port for GNU/Mach", and "worked with GNU/Hurd a lot"

<solid_black>there are these two threads on debian-hurd that mention some of their progress https://lists.debian.org/debian-hurd/2009/11/msg00091.html https://lists.debian.org/debian-hurd/2010/03/msg00009.html

<solid_black>I then found this page http://web.archive.org/web/20101220201149/http://www.ksyslabs.org/projects/gnu-mach/sources

<solid_black>but either Bitbucket has fully dropped their repo, or I don't understand hg enough, but I never found the actual sources

<solid_black>here are the slides http://web.archive.org/web/20121220024633/https://www.slideshare.net/sartakov/ss-8140660 -- the slides themselves don't load, but you can Ctrl-F the text for "pfinet"

<azert>solid_black: yes timer and gic is detected correctly

<azert>That’s why I thought about linux style access to memory mapped registers

<azert>In regards to heavy threading, I think you are mistaken

<solid_black>how so?

<azert>First, gnumach has very light threads

<solid_black>by "heavy", I don't mean that they are inefficient

<solid_black>I mean that they are heavily used, in other words, used a lot

<azert>i think that’s the way Mach was meant to work

<solid_black>not necesserily

<solid_black>what I'm talking about is

<azert>A port is the equivalent of a gate in x86 terms

<solid_black>whenever there's an RPC that can block, the current Hurd design is to run it in its own thread and actually block that thread

<solid_black>and libports just spawns more threads as needed to handle incoming RPCs

<azert>That’s how it was meant

<solid_black>but an alternative is to use MIG_NO_REPLY (if using MIG), just save the state of the request somewhere, and reply to it later

<solid_black>that's how my epoll server is written for example

<solid_black>that's also how the in-Mach device driver framework works

<solid_black>when you device_read(), it doesn't block your thread in the kernel, it just stores the "io request" for later

<solid_black>with language-assisted async/await, we could get the best of both worlds

<solid_black>you write the code linearly, as if it's blocking

<azert>Its syntactic sugar over callbacks

<solid_black>but it doesn't spawn more and more threads, it spawns lightweight tasks

<solid_black>it is, yes

<solid_black>all languages are syntactic sugar over machine code :)

<azert>No I think C was designed to rule the world

<azert>But that’s a different discussion

<damo22>why arent all compiled languages libraries for C?

<solid_black>hi damo22!

<azert>Hi

<damo22>hello

<solid_black>the timer is configured with system registers, not memory accesses

<Pellescours>hi

<solid_black>those could use barriers too I guess, but I doubt that should cause issues

<azert>solid_black what about the gic?

<solid_black>so it might just we're not turning on the GIC, yes

<azert>You need to enable the interupts

<azert>This is my theory now

<azert>That we fail to do that maybe because of barriers

<solid_black>does the thread-state test fail? that one shouldn't depend on timers

<azert>I think it fails with a panic

<solid_black>oh?

<azert>We can start to look at that

<azert>If it’s more easy

<solid_black>we can do bot

<solid_black>h

<solid_black>do you have any idea how keyboards... work? like how would one write a driver for a keyboard?

<solid_black>it's probably connected over USB?

<solid_black>do we need a separate driver for USB? is that a part of rump?

<Pellescours>yes there is a driver for usb in rump

<solid_black>with it just work on arm?

<Pellescours>probably, but even for x86 we didn't wired the usb stack (except disks on USB)

<Pellescours>damo22 could probably say more on this, he knows rump very well

<damo22>the problem is, currently rump cannot separate the usb stack in half, we need a driver for just the controller, and have other peripheral drivers attach to the controller driver

<damo22>in netbsd kernel, everything is monolithic, so it does not matter

<azert98>keyboard is one of the reasons yeasterday I said we need apis to do it the microkernel way

<azert98>I'll just make a monolithic device server

<azert98>damo22 is right

<azert98>in any case, the most important devices are the sd card and the ethernet

<damo22>we cannot have monolithic usb stack, otherwise all usb drivers need to be part of the usb driver

<solid_black>yes

<azert98>of course a part the uart

<solid_black>and after those two, keyboard and display

<azert98>damo22 exactly it will be basically everything in

<solid_black>and support for those in the Hurd console

<damo22>anything that attaches via usb will need to be in the usb driver

<damo22>but thats not very hurdish

<solid_black>azert98: I think what damo22 is saying is that this monolithic setup is not a good idea

<azert98>damo22 I know it sucks, but that would be a temporary solution with two different outcomes

<azert98>one outcome is that it develops in a novel usb framework

<azert98>the other outcome is that you get a monolithic device server with plugins

<Pellescours>damo22: to split rump usb stack, it's just compilation, or do you think the code also need to be patched?

<azert98>if you design a novel hurdish usb stack , I'd love that

<damo22>there is something called ugenhc and ugen

<damo22>this may do what we need

<damo22>but the code needs to be updated

<damo22>its like a generic usb host controller and generic usb driver

<damo22>i think it uses ioctls to configure the devices

<azert98>I think we need to look into libchannel

<azert98>if we are going the rump way, plugins are probably easy to implement and you could reuse netbsd modules

<damo22>what i am not sure about, is how do you get the host controller driver to probe a device that is not compiled in

<azert98>it's a problem that netbsd people already solved I guess

<damo22>not really, they solved it by making the kernel monolithic

<damo22>the main developer of rump started making it split but im not sure if he finished it

<damo22>i saw some chatter about it on the old rump mailing list

<azert98>splitting it in naive ways might be very inefficient? I think that libchannel was being designed to avoid copies

<solid_black>copies of what?

<azert98> https://www.gnu.org/software/hurd//hurd/libchannel.html

<azert98>copies between elements of the stack

<azert98>copies of the data

<solid_black>that wiki page reads like it avoids extra rpcs, not copies

<solid_black>you don't copy anything anyway (if you design everything right), Mach does the VM CoW thing

<azert98> https://lwn.net/Articles/169961/

<azert98>it's linked in the wiki

<azert98>I think it was the original idea

<damo22>what we want is /dev/ugenhc and the drivers for each peripheral open this to get access to the host controller

<azert98>btw, this is the output of thread-state test:

<solid_black>the lwn post is about avoiding data copies, yes

<solid_black>damo22: I'd be glad to help you design the RPCs for a more Hurdish USB stack

<solid_black>if you can explain how USB works / what it needs / etc

<damo22>i wish i knew

<solid_black>:D

<solid_black>also: virtio

<azert98> https://paste.debian.net/1313868/

<solid_black>does anybody know anyting about virtio?

<damo22>virtio already works in rump but due to compilation issue i commented it out

<solid_black>will it work on arm?

<damo22>maybe if you can compile librump on arm

<Pellescours>damo22: with uodate to netbds10, it may be fixed

<damo22>Pellescours: do you mean virtio?

<Pellescours>yes

<damo22>ok

<solid_black>how does it (librump) even access hardware? does it mmap /dev/mem?

<damo22>it uses pciaccess for pci

<Pellescours>i tried yesterday, only 2 patches need to be updated (the one that touch to build.sh) but the VM frooze during compilation so I did not continue

<solid_black>ok...

<solid_black>how does libpciaccess work then?

<solid_black>does it mmap /dev/mem?

<solid_black>does it have an arm backend?

<damo22>no, device_open(mem)

<solid_black>sure, I meant that

<solid_black>ok

<solid_black>how does it learn about PCI

<solid_black>'s existance

<solid_black>as in a memory address

<damo22>pci-arbiter

<solid_black>no, pci-arbiter is on a Hurd system

<solid_black>I imagine on x86, there must be some fixed I/O port for PCI

<solid_black>that it just probes?

<damo22>oh ok

<damo22>yes

<solid_black>but on ARM, it should be reading the device tree?

<solid_black>does it do that?

<damo22>PCI_CNF1

<damo22>theres a fixed io port range for pci on x86

<damo22>if you need to access pci on arm, you probably need to add it to libpciaccess?

<solid_black>hm, looks like azert98's board doesn't have PCI at all, is that right?

<azert98>solid_black: I think even of arm PCI is an independent beast

<azert98>often not there indeed

<solid_black>usb@1c19000 for example is attached directly to the "soc", no PCI in sight

<damo22>solid_black: that probably simplifies things

<solid_black>that's his particular board, doesn't mean we don't need PCI support overall

<damo22>we could add iommu support to pci-arbiter

<solid_black>is iommu the same thing as smmu?

<solid_black>azert98: that's a question for you I guess

<damo22>i started decoding the DMAR acpi table to read out the configuration

<azert98>yes an smmu is an iommu in the arm world

<damo22>i feel like we are reinventing the linux kernel one module at a time

<azert98>acpi, another thing replaced by the device-tree in the microcontroller world

<damo22>here is a suggestion: a driver "kit" that is monolithic like the linux kernel but fans out into separate modules for peripherals, that map 1:1 to linux driver modules

<azert98>isn't this what rump is?

<damo22>not quite, rump is separated

<damo22>rump is a collection of unikernels

<youpi>you can't really avoid that

<youpi>since each driver depends on its kernelish runtime

<youpi>I don't think it's really that much a problem, provided you can prune whatever the driver doesn't need

<damo22>yes, can we have just one copy of a linux kernelish runtime, and connect all the modules to it like plugins

<azert98>solid_black: you see the pmap error https://paste.debian.net/1313868/

<youpi>(I don't think taking Linux as a base is long-term-prone, it has a way too-instable internal API)

<youpi>damo22: but that'd be hairy

<solid_black>azert98: I'm looking through the log, haven't yet reached the pmap panic

<youpi>e.g. asking for interrupt handling

<youpi>I don't see the benefit of stuffing that into yet another translator

<azert98>one could think about running a linux virtual machine with a linux module to serve the hardware to gnumach supervising it

<youpi>when the driver can directly ask mach

<youpi>by "runtime", I basically mean sort-of-C-library

<youpi>not much to share anyway

<youpi>once one is in a separate process, its memory management would be separate

<youpi>same for thread pool etc.

<youpi>that what I mean by "runtime"

<damo22>ok

<azert98>you would have better chances to make a gnumach hypervisor and use it to run linux

<damo22>why not run linux to provide drivers?

<solid_black>just running Linux in a VM is the sad end that the other microkernels end up on

<youpi>well, that's dde

<youpi>and we know how much people managed to maintain it

<youpi>again, really, I'd advise forgetting about linux, and just focus on rump

<youpi>it's the same benefit, for much easier maintenance

<youpi>solid_black: not even, all such projects I have seen didn't manage to maintain long-term

<damo22>but rump is semi-unmaintained

<youpi>gotta run to a meeting

<youpi>that's better than unmaintained

<youpi>and it has been so for quite some time already

<damo22>ok

<Pellescours>did someone played with SMP recently ? I would lije to know if it's stable enough to for compilation (of rumpkernel for example)

<solid_black>speaking of SMP

<solid_black>I tried the Debian package

<solid_black>and it didn't seem to actually enable SMP for me

<solid_black>nproc still printed 1

<Pellescours>the AP cpus are ine another Pset

<damo22>nproc should be N

<damo22>did you run with -smp N

<solid_black>how does nproc(1) even work?

<solid_black>I ran it on vbox, on a multi-core VM, yes

<solid_black>do I still need the smp(1) tool?

<damo22>i havent tried the debian smp kernel yet

<damo22>Pellescours: if you dont enable the APs with smp utility, the smp kernel should be able to compile anything on the BSP just fine

<Pellescours>yes, i know

<Pellescours>i want to be able to use the power of 16 cores for compilation

<damo22>probably will crash

<Pellescours>x)

<azert98>you need 16x the memory

<Pellescours>i can reduce the number of core but juste having 2 or 4 core will reduce time a lot

<damo22>need solid_black's patch for vm_page

<damo22>and maybe more fixes

<solid_black>I thought that was already in Debian?

<damo22>the HACK HACK... thing?

<solid_black>yes

<solid_black>the new version that I posted recently

<damo22>ok i didnt notice

<damo22>thats good then

<Pellescours>i already know that stressing disk or memory make the userland freeze completely with no smp.

<damo22>Pellescours: that is probably because you need swap space

<azert98>solid-black: without tracing https://paste.debian.net/1313869/ if it helps

<damo22>i have 8GB swap disk

<solid_black>oh

<solid_black>exception_raise(1 102 0x00fffffffffd98)

<solid_black>102 is EXC_AARCH64_AL_SP

<solid_black>misaligned stack pointer

<solid_black>and indeed I never aligned it in the test

<solid_black>that shouldn't panic Mach though

<azert98>better fixing the panic before?

<solid_black>also it tells you the misaligned sp value in the subcode, look how cute that is

<azert98>if you fix the misaligned stack, chances are that the panic will disappear

<solid_black>0x00fffffffffd98 is not 16-aligned indeed

<solid_black>yeah let's fix the panic first

<FunkyNeonSamurai>Hi everyone! I'm a master's infosec student interested in system security and I found about hurd ng recently. I'm interested in working on systems security research and I'm interested to find out about hurd more.

<AwesomeAdam54321>FunkyNeonSamurai: Do you have any questions about HURD NG?

<Pellescours>welcome

<solid_black>so what's happening is it's tearing down all mappings for the task

<solid_black>'cause the task is being terminated

<FunkyNeonSamurai>AwesomeAdam54321: not immediately, but i'm planning to get it up and running soon-ish. If there's anything that I can contribute, it'll probably be to security stuff,

<FunkyNeonSamurai>Pellescours: thanks!

<azert>FunkyNeonSamurai What do you mean about security?

<azert>You should know that you are never safe

<solid_black>for example, not using executable stacks :)

<solid_black>azert: can you add some tracing to pmap_page_protect?

<solid_black>like addr & prot at the start, and then pmap and v in the loop before the pmap_walk call

<azert>Yes, what exactly?

<azert>Okay

<azert>I’ll do it later

<solid_black>I guess entry and *entry after the pmap_walk call too

<solid_black>but look at how it actually faulted on misaligned stack pointer, and Mach produced the expected exception and sent it to the other thread, and the other thread logged it and decided to do nothing with it, and Mach killed the task

<solid_black>this is so cool :)

<Pellescours>is it aach64 feature only, or do all arch benefits from this improvements?

<solid_black>stack alignment enforcement is an aarch64 hardware feature

<solid_black>things like fixing executable stack support, all arches benefit from

<azert>solid_black: pmap page protect phys=0x000000007f814000, prot=0x00000000 :

<azert> pmap walk in pmap=0xffffffffffffffff, v=0xffffffffffffffff ; ../aarch64/aarch64/pmap.c:1075: pmap_page_protect: Assertion `kr == KERN_SUCCESS' failed.panic ../kern/debug.c:107: Debugger: Debugger invoked, but there isn't one!

<solid_black>pmap is -1???

<azert>looks like

<solid_black>something is extremely broken about pv tracking

<solid_black>you could maybe add some tracing into pv_link and pv_unlink

<p4r4D0xum>Can anybody tell me where can I find git-new-workdir? Can't find it in /usr/share/...

<solid_black>p4r4D0xum: /usr/share/doc/git/contrib/workdir/git-new-workdir

<p4r4D0xum>solid_black: yes, it ain't there.

<p4r4D0xum>oh, my apologies...

<p4r4D0xum>solid_black: Thank you.

<solid_black>you're welcome, though I have no idea what that has to do with the Hurd :)

<p4r4D0xum>solid_black: incubator. :)

<p4r4D0xum>whois ThinkT510

<p4r4D0xum>ups

<p4r4D0xum>:)

<ThinkT510>I am me

<p4r4D0xum>ThinkT510: I get that. :)

<p4r4D0xum>Have T530 here.

<azert>solid_black: here it is https://paste.debian.net/1313903/

<p4r4D0xum>Does T510 have SSD as well?

<solid_black>hmm, so it unlinks a bunch of pvs *before* calling pmap_page_protect?

<solid_black>that's pmap_remove, right

<solid_black>we do have pv link pmap=0xffff00007f81cf90, pa=0x000000007f814000, v=0x0000000000400000 before

<solid_black>then it is unlinked

<solid_black>which apparently leaves the pv in a bad state?

<solid_black>it should've set pv_h->pmap = PMAP_NULL; pv_h->va = 0;

<solid_black>not -1

<solid_black>I can not reproduce the panic in qemu if I synthesize an unhandled exception

<solid_black>so it has to be due to something that qemu does differently from real hardware

<solid_black>or due to some difference in your setup vs mine, such as memory layout

<azert>solid_black: I’m wondering if the difference is not in the values of uninitialized memory

<azert>Like I can phantom real hardware to have 0xffffffff and qemu to have 0x00000000

<azert>Then a real bug would stay unnoticed on qemu and emerge on real hw

<azert> https://bugs.launchpad.net/qemu/+bug/1180923

<azert>Apparently people

<azert>at qemu don’t care about such an obvious bug https://gitlab.com/qemu-project/qemu/-/issues/1133

<azert>Fine, maybe I’m not even right

<azert>solid_black: I have another more valid theory

<azert>pmap_init allocate pv_head_table by kmem_alloc_wired, addr is 0xffff000081600000

<azert>Kernel is at 0xffff000040000000

<azert>I have 1gb of memory

<azert>Make the math..

<azert>It’s out of memory

<azert>Unless this is just a virtual address.. ??

<azert>Thing is that if I memset addr to zero, the controller crash

<azert>Indicating that there might be an issue

<saravia`>azert <-- you know how to do a patch for that <-- because I could send the request in hyperbola linux distro <-- for the qemu PKGBUILD repository <-- maybe that will be help to hyperbolabsd OS building

<azert>saravia`: not that is an hard to fix bug, and I don't think it is the culprit here

<gnucode>howdy friends!

<anatoly>solid_black https://bitbucket.org/blog/sunsetting-mercurial-support-in-bitbucket here we go why you don't understand hg :-D but i don't understand why it couldn't be archieved and preserved by bitbucket (i mean I understand, it's money)

<anatoly>So he's my updated original question: is it the first arm64 board running GNU Mach micro-kernel? :-D

IRC channel logs

2024-04-12.log