IRC channel logs


back to list of logs

<damo22>nice thing about this hw is its fanless
<almuhs>damo22: i had problems with rumpdisk in many Thinkpad models. In the T410, if i remember well, the installer in rumpdisk mode didn't detect the harddisk. Similar in T60 and other models. In some cases, the installer detected the harddisk but not the CD drive
<almuhs>so it's possible that rumpdisk has any issue in physical hardisk
<youpi>damo22: perhaps it's the access to the hdd which is slow then
<damo22>youpi: it is using /dev/sda which is a dedicated msata SSD
<damo22>i think its booting with rumpdisk too
<youpi>ok but I mean possibly the disk driver itself gets slow for whatever reason
<youpi>right now I'm seeing that using the qemu ahci driver is quite faster than letting qemu pick up its default driver
<damo22>i dont understand
<youpi>kvm -drive file=foo -> slow
<youpi>kvm -device ahci, ... -drive file=... -device ide-hd,... -> fast
<damo22>qemu-system-i386 -M q35,accel=kvm -m 4096 -net user,hostfwd=tcp::8888-:22 -net nic -curses -hda /dev/sda
<damo22>shall i shut it down and retry with different params now?
<youpi>with q35, I do get a fast speed
<damo22>yes i always run with that param, the -M pc is slow as
<damo22>which is default i think
<youpi>I don't see a good reason why it should be that slow
<youpi>looks like a bug somewhere
<youpi>oh, btw, you want -cpu host
<youpi>so libc can use sse etc.
<damo22>oh ok
<almuhs>during the hurd booting in smp mode, when the boot freeze, my fan speed up, like the cpu make a big effort. This usually happens when a process enter in a infinite loop. Could gnumach are entering in a infinite loop in some scenarios?
<almuhs>i've just found an harddisk error running Debian GNU/Hurd with -smp 2
<damo22>yes its not ready, something is broken
<damo22>youpi: i cant use -cpu host or hurd hangs
<damo22>however i have switched off rumpdisk for now
<damo22>perhaps it will be faster with gnumach ahci
<damo22>almuhs: yes i have seen that error, its probably because an interrupt is lost or something
<youpi>such hang is not supposed to happen, it's probably worth investigating
<almuhs>anyway, i only got a shell with smp from time to time. Around 99% attempts fails the booting in smp mode, hanging during the boot
<damo22>almuhs: did you use my experimental patch?
<almuhs>no, i'm not
<damo22>almuhs: see top commit of this
<damo22>it makes most of the hangs go away
<damo22>and runs faster
<almuhs>but that is a ugly patch, it's not?
<almuhs>this patch don't solve the origin of the problem
<damo22>well its not too bad, it just makes more threads go on the run queues
<damo22>instead of dispatching direct to idle processor
<almuhs>i understand
<damo22>in theory it should be slower yes, but for some reason its faster
<almuhs>but the speed is not as important problem than the hangs
<damo22>yes most of the hangs are also gone with this patch
<damo22>but i dont understand why
<almuhs>"../include/mach/std_types.defs", line 32: overriding previous definition of char
<almuhs>oh! "configure: WARNING: unrecognized options: --enable-apic, --enable-ncpus, --disable-linux-groups"
<almuhs>i'm trying to compile from your repository's sources
<damo22>autoreconf -fi first
<damo22>my sources are master + one commit
<damo22>you may need to check out "fixes" branch
<almuhs>i'm using debian-rumpdisk-boots
<damo22>no that is super old
<almuhs>oh, bad branch then
<almuhs>testing with -smp 4 , hangs just before ask the username
<almuhs>ops, wait
<almuhs>now copied from correctly path
<damo22>yes it does hang randomly but works more often than without the patch
<almuhs>ok, now i got successful login with -smp 4
<almuhs>what command can i use to check what processor executes each thread?
<damo22>idk but you can install stress and run stress -c 4
<damo22>it will hog 4 cpus
<damo22>then you can run top and check
<almuhs>all threads seems to runs in cpu 0
<damo22>i doubt it, there is 100% load on 4 separate processes
<almuhs>how can i show all process inclusing the system process?
<almuhs>"P" column
<youpi>reporting this is possibly just not even supported
<almuhs>we added this support 3 years ago
<almuhs>i patched it
<almuhs>in upstream
<almuhs>in gnumach and hurd. I added a new field in the scheduler struct to add the last_thread, and i patched two hurd servers to recover this new field and fill the process table with the information
<youpi>but is that where top reads it?
<almuhs>i think yes, but i'm not sure
<almuhs>we test it with ps command
<youpi>ps is not top
<almuhs>yes, but they can take the info from similar sources, it's not?
<youpi>ps uses libps
<youpi>top reads /proc
<almuhs>this was the commit
<almuhs>and that from the gnumach side
<youpi>ok, so that should be working
<almuhs>how can i check the last processor using ps?
<youpi>possibly it's not even implemented
<almuhs>in that time, you told me a ps arguments to check it
<youpi>but I just cannot remember such thing by heart
<almuhs>ps has a TH column, but in blank
<almuhs>ok, it seems that ps command in hurd is more limited than in linux
<almuhs>it lacks some columns
<almuhs>ps -e -o pid,psr,comm
<almuhs>ps: psr: Unknown format spec
<almuhs>i have to go sleep. bye
<damo22>i dont see that defined anywhere in hurd
<damo22>AC_CHECK_MEMBERS([struct thread_sched_info.last_processor] does this do some magic?
<damo22>When member belongs to aggregate, define HAVE_aggregate_member (in all capitals, with spaces and dots replaced by underscores)
<damo22>seems yes
<damo22> integer_t depress_priority; /* priority depressed from */
<damo22> integer_t last_processor; /* last processor used by the thread */
<damo22>#define THREAD_SCHED_INFO_COUNT \
<damo22> (sizeof(thread_sched_info_data_t) / sizeof(natural_t))
<damo22>is natural_t equivalent to integer_t ?
<damo22>mach/i386/vm_types.h:typedef int integer_t;
<damo22>mach/i386/vm_types.h:typedef unsigned int natural_t;
<damo22>i guess theyre the same size at least
<damo22>+ * Strictly speaking, this is wrong. The PIRQ should be routed
<damo22>+ * to *both* the I/O APIC and the PIC, on different pins. The
<damo22>+ * I/O APIC has a fixed mapping to IRQ16-23, while the PIC is
<damo22>+ * routed according to the PIRQx_ROUT configuration. But QEMU
<damo22>+ * doesn't (yet) cope with the concept of pin numbers differing
<damo22>+ * between PIC and I/O APIC, and neither does the in-kernel KVM
<damo22>+ * irqchip support. So we route to the I/O APIC *only* if the
<damo22>+ * routing to the PIC is disabled in the PIRQx_ROUT settings.
<damo22>+ *
<damo22>+ * This seems to work even if we boot a Linux guest with 'noapic'
<damo22>+ * to make it use the legacy PIC, and then kexec directly into a
<damo22>+ * new kernel which uses the I/O APIC. The new kernel explicitly
<damo22>+ * disables the PIRQ routing even though it doesn't need to care.
<damo22>qemu ioapic/pic routing
<damo22>maybe that could explain some strangeness with interrupts?
<damo22>qemu @ 886e0a5f31bf3d40dd8d9199674a4bad64942fde
<damo22>solid_black: hi
<solid_black>hi damo22!
<damo22> what do you think of this?
<solid_black>not that I know anything about sound...
<damo22>i do
<solid_black>but as I understood youpi, we want to use PipeWire, like GNU/Linux distros do nowdays
<solid_black>which uses alsa I believe?
<damo22>pipewire is a stack that sits on top of ALSA
<damo22>its not ideal
<damo22>ALSA has flaws
<solid_black>so either we should support alsa, or PipeWire would have to gain a Hurd-specific backend
<damo22>i think we should expose a JACK api
<damo22>pipewire can sit on top of that if needde
<Arsen>I don't think that makes sense
<Arsen>JACK also has some of the flas noted in that email AFAIK
<Arsen>plus, it's quite high level
<damo22>yes, users dont need to open() read() write() to a sound card
<damo22>video doesnt allow that
<damo22>we need to expose a callback mechanism
<damo22>that is the problem with linux sound
<damo22>its completely backwards
<damo22>if JACK api is missing some things, we can extend its api
<solid_black>so that linked wiki page looks like a list of things that people would like to improve in a potential alsa successor
<solid_black>kind of like that "X12" wishlist
<Arsen>that doesn't require implementing jack, something more fit to mach seems to make sense
<damo22>solid_black: yes
<damo22>reason: i dont want to break all audio apps
<damo22>we need to expose an existing api
<solid_black>does pw support a jack backend?
<damo22>but the internals can be better designed
<solid_black>I thought it only implemented jack compatibility for clients
<damo22>no it does both iirc
<solid_black>keeping up with the trend, should we schedule a call with Wim Taymans? :D
<damo22>he thinks polling a fd for IO is not bad for audio
<solid_black>is there an intro to the jack api that you would recommend?
<solid_black>so I can see what it is that we're talking about
<damo22>basically, you open a connection to a jack server, then you register some ports, and register a callback that you will be using to fill the buffers with, and start the server.... jack decides WHEN to call your callback to retrieve the audio data
<solid_black>let me ask a different question I guess
<damo22>its a pull model
<solid_black>we want to reuse some existing drivers for sound cards, don't we? rump?
<solid_black>what API do those drivers export? alsa?
<damo22>i was thinking to reuse ALSA low level drivers
<damo22>they are a bunch of function pointers
<damo22>implemented for each card
<damo22>but rewrite the pcm streaming
<Gooberpatrol66>what's the difference between the way jack works and pipewire works
<solid_black>so would it make sense to
<solid_black>have a soundcard device driver run in its own translator (naturally)
<Gooberpatrol66>i remember when i used jack i could only use 1 soundcard at a time and pipewire allegedly fixed that
<damo22>yes, jack supports one sound card at a time
<solid_black>and communicate with the PW deamon using a bespoke Mach IPC protocol (sound.defs perhaps) that is neither alsa or jack
<damo22>its one of its flaws
<solid_black>and then the apps/clients speak PA, or PW's natuve protocol, or Jack (or alsa?) to pw
<damo22>pulseaudio is garbage
<solid_black>I'm not suggesting to use the actual pulseaudio
<damo22>ah ok
<solid_black>but its protocol is what most clients speak, to the PW daemon, or rather the pw-pa bridge
<Gooberpatrol66>would the drivers use pipewire code to speak jack? (because pipewire implements jack API)?
<Gooberpatrol66>i think pipewire has its own native protocol but nobody uses it
<damo22>i was thinking we could write a driver that exposes a filesystem node like /dev/snd that has some kind of API that can be implemented as a jack backend
<solid_black>(also I cannot speak to whether pa is garbage, but Lennart is one of the smartest, most talented people I know about, despite all of the shit that gets said about him)
<damo22>the main flaw with ALSA is that transfer of samples occurs by arrival of audio IRQ
<damo22>its a limiting design
<damo22>the driver needs to be rewritten
<solid_black>so is it true that jack also has a protocol that clients speak to a central daemon? and that daemon uses alsa to talk to the device?
<damo22>that part of jack is solid
<solid_black>then I don't understand how jack is going to be involved in your plan
<damo22>implement a jack backend
<damo22>so applications dont need to changre
<damo22>they can continue connecting to jack
<solid_black>you want individual clients/apps to use the jack api/protocol?
<damo22>they already do
<Gooberpatrol66>have you ever looked at sndio? it has some neat features
<solid_black>no they don't?
<damo22>many audio applications
<solid_black>pro-audio apps do
<solid_black>typical desktop ones use pa
<damo22>theyre the only ones i use
<solid_black>so you don't want PW to be involved in any of this?
<damo22>it can sit on top if you really need something non-jack
<solid_black>or do you want PW to be there only for providing PA compatibility for those clients that don't speak jack natively?
<solid_black>and you're saying that pw already has a jack backend
<damo22>i believe it does yes
<solid_black>how would the jack daemon talk to the device in your plan?
<damo22>open /dev/snd and use RPCs to talk a hurdish api that implements all the things jack needs
<solid_black>so the daemon would need a Hurd-specific backend
<solid_black>but otherwise it'd be an upstream jack daemon?
<damo22>yes we would need to implement a hurd jack backend for jack
<solid_black>and if someone wouldn't want to use Jack, they could also implement a backend for the same Hurdish protocol into pw?
<Arsen>if I follow, you mean to implement not jack, but a backend for Jack to use, and then for pw to use indirectly via jack?
<Arsen>why not just skip the middle man
<Arsen>I like jack but I don't follow the design here
<damo22>you can also do that Arsen if you wish
<damo22>but that part is already done
<Arsen>oh, I see
<damo22>pw can run on top of jack already
<damo22>so we dont need to do any work for that path
<solid_black>~Linux~ GNU/Hurd is about choice! :D
<solid_black>IIRC was in fact about audio
<Arsen>... yes, but jack can't run on top of Hurd (yet)
<solid_black>it would really help if you said "jack daemon" and not just "jack" (if the daemon is what you mean)
<damo22>yes, we need a sound driver
<Gooberpatrol66>"pw can run on top of jack"
<solid_black>'cause I get a bit lost trying to follow
<Arsen>so if the goal is running pw, porting jack rather than pw doesn't seem to be productive
<Gooberpatrol66>you mean pw implements jack
<damo22>yes pw implements jack api
<solid_black>see, Gooberpatrol66 got a bit lost too
<Arsen>I don't, above someone said that pw implements a jack based backend
<damo22>it can do both
<Arsen>as in, jack provides the audio devices
<solid_black>pw, as far as I've heard, implements the jack client library
<damo22>alsa -> jack server -> pw -> speakers
<Arsen>it does that indeed, but apparently it also can be a jack client
<solid_black>which then speaks the pw native api to the pw deamon, but this i'm not sure about
<damo22>or alsa -> pw -> speaekrs
<damo22>or alsa -> jack -> pw -> speakers
<damo22>alsa -> pw -> jack api -> jack client
<damo22>what we want to do is :
<solid_black>can we have /dev/snd both support an alsa api (which is read/write/ioctl on the device node, right?), and the additional Hurdish API?
<damo22>hurd sound -> jack api -> jack client (or pipewire -> pw client)
<damo22>and one day someone can do hurd sound -> pw api
<damo22>i dont want alsa api anywhere
<solid_black>the point of exporting the alsa api is that one could use jack or pw daemons even before we'd implement Hurd-specific backends into them
<damo22>i think its a bad idea
<damo22>the whole point of rewriting alsa is to fix its flaws in its api
<damo22>for example, i want to add port names
<Gooberpatrol66>pipewire is designed to transfer data between containers, which could be important for hurd, given how compartmentalized it is
<damo22>ALSA api does not export the port names of each i/o
<Gooberpatrol66>(and it works for any type of data, not just audio)
<damo22>also the device enumeration is not unique in alsa
<damo22>so you unplug a sound card and plug it again and you might get a different card number
<damo22>its bad
<damo22>ALSA also supports OSS emulation which needs to die
<damo22>i want to remove all that junk
<damo22>i want an audio framebuffer
<damo22>with accurate latency reporting
<Gooberpatrol66>so would a sound card driver emulate a jack daemon and then speak to another jack daemon that acts as the sound server? in your idea?
<Gooberpatrol66>one daemon for each sound card?
<damo22>no, a sound card driver would expose a set of RPCs that something like a jack daemon would need
<damo22>and we implement that for jackd
<damo22>it should be a trivial port if the api matches roughly
<damo22>the jackd part i mean
<damo22>the hard part is making the driver
<damo22>so jackd would open(/dev/snd) and call rpcs that it needs to control it
<damo22>the audio data does not flow through /dev/snd
<damo22>it just registers a callback with a pointer to some buffer
<solid_black>what sort of callback and a buffer, how would that map to the Mach RPCs?
<solid_black>would it not still be a Mach message carrying the buffer contents?
<damo22>i dont think we can pass audio data as a mach message, wont it be too slow?
<solid_black>how else do you want this to work? shared memory?
<solid_black>or, would jackd mmap the device buffer directly? (is that possible?)
<damo22>yes, shared memory to a small ringbuffer?
<damo22>or mmap
<damo22>the audio driver would mmap the device buffer directly and somehow we need to share that with the caller
<solid_black>the caller being jackd?
<damo22>no the audio driver would start and stop the card
<damo22>we just need to fill its buffer somehow
<damo22>the audio driver would know when to do this
<damo22>scheduling a writer and a reader probably
<damo22>based on timers
<damo22>the arrival of the audio IRQ tells you when the hw pointer is at the end of the cycle
<damo22>but you dont just copy the buffer when this happens
<damo22>(like alsa does)
<damo22>you use the timestamps of arrival of the audio IRQs to calibrate a delay locked loop
<damo22>then you can ask the question : where is the buffer pointer now
<damo22>at any time you want in between the audio irq
<damo22>and use some kind of timer interrupt to schedule the transfer when you want
<damo22>depending on the requested latency
<damo22>this all happens in the audio driver
<damo22>but its controlled by jackd, and the audio driver will call the callback provided by the user application
<solid_black>again, what does 'call the callback' mean here
<solid_black>if we're talking cross-process
<solid_black>it must be a Mach RPC, no?
<damo22>yeh i havent figured that part out yet
<damo22>the audio driver basically writes and reads zeroes until you fill it with some audio data
<damo22>and if the callback blocks for too long, and cant deliver the audio in time for the next cycle, the audio driver drops the frame and you get silence
<damo22>unless the audio driver is in the same process as jackd
<damo22>somehow jackd gets access to a mmap'd buffer
<damo22>jackd handles all the routing between different audio applications
<damo22>it creates a process graph
<damo22>cant one process call an RPC that mmaps the same buffer as another process and return a pointer to it?
<damo22>i dont know how shared memory works
<damo22>but user audio basically takes a buffer of floats and processes them into another buffer of floats
<damo22>solid_black: jackd solved that problem somehow already, because jackd supports multiple processes connecting to it and registering callbacks with jackd so jackd can call them when it needs to fill the buffers
<solid_black>so another thing to discuss would be the graphics
<solid_black>we all want Wayland, don't we?
<damo22>hi aze
<solid_black>Wayland itself, the protocol and the supporting libraries, might have been ported already
<solid_black>hello azert
<azert>I was reading damo22 plans for audio on the logs
<solid_black>the clients (such as gtk) should be buildable with wayland support, should the distros ship wayland
<solid_black>the issue is with compositors & their backends
<solid_black>I've ported Owl, but Owl uses GNUstep as its backend, and that relies on X11 again
<solid_black>this is cool and all, but it's not a solution for graphics on the Hurd
<damo22>we need mesa
<solid_black>Weston, the original reference Wayland compositor, has removed its framebuffer backend in 2022
<solid_black>it might be possible to patch it back
<solid_black>there is Mesa, with software rendering
<solid_black>Linux's DRM is the API everyone uses
<solid_black>is there any chance we could get that? what does rump do?
<Arsen>fwiw managarm runs modern weston
<Arsen>so getting enough DRM to run isn't impossible
<Arsen>just classic ioctl ugliness
<azert>Maybe one can use a rump drm implementation?
<solid_black>does rump have a drm implementation?
<damo22>we should probably port it if not
<solid_black>port what? drm off of linux?
<azert>Netbsd has drm
<damo22>i think someone is rewriting it
<damo22>because netbsd drm is based on linux drm but is somewhat borked
<damo22>last i heard
<damo22>i was working on netbsd's audio driver to introduce a DLL to provide more accurate buffer pointers
<azert>Regarding audio, one could maybe write drivers as Jack clients?
<damo22>jack client?
<azert>Not very hurdish, since it won’t be expose to the file system, but is there a reason not to?
<damo22>no no, thats not how it works
<damo22>jackd does not know how to drive cards directly
<damo22>it uses ALSA
<azert>But the jack api works for both input as well as output, right?
<azert>So if the client knows how to drive a card, why wouldn’t it work?
<damo22>its not designed to work that way
<damo22>jackd cant read or write to a card it doesnt have a backend for
<azert>Oh fine, was just an idea
<azert>It would just read/write to a client
<azert>That happens to be an userspace driver
<damo22>oh wait, i kinda see
<azert>But maybe I just don’t know jack well enough
<damo22>thats a clever idea, but inverts the concept of a playback port
<damo22>the driver connects to jack
<damo22>and exposes all the ports that the card supports
<damo22>but its a bit weird
<damo22>the card would show up as a client
<damo22>but the process graph probably would not function correctly
<damo22>because jackd would try to write to the backend
<damo22>and theres nothing there
<damo22>i want to reuse ALSA low level functions to deal with the card specifics
<damo22>but implement the streaming from scratch
<azert>I think you can connect clients to clients directly already, excluding the backend from the picture
<azert>The backend would just not be there
<damo22>yes but the process graph is computed from the root
<damo22>i dont think it would operate as expected
<damo22>i think the whole point of the graph is to minimise context switches between threads implementing each client's audio processor
<damo22>the client for the card would not actually process anything, it generates samples and consumes samples
<azert>It’s a leaf of the graph
<damo22>but can it be a leaf and a client ?
<azert>I don’t see why not but I’d ask Jack developers
<damo22>actually i implemented this already for QubesOS
<damo22>i connected jack as a client to another jack instance on dom0
<azert>solid_black: I’ve looked at sys/external/bsd/drm2 in netbsd tree. They just ported the Linux drm, if they can do that probably it means it is portable?
<solid_black>that it's portable in theory we know, it's also been ported onto FreeBSD IIRC
<solid_black>the question is, will we get it "for free" with rump, or would we also have to invest into porting it
<azert>How hard is it to create a new rump module?
<solid_black>damo22: ^
<azert>I think he sleeps
<solid_black>ah, right
<Gooberpatrol_66>rust was merged btw (yay)
<youpi>Gooberpatrol_66: yes but there are missing changes in a few repos before it can be built at all
<youpi>some of them are waiting for a CI to be integrated, which means waiting for libc to be released
<youpi>hopefully that shouldn't take too many weeks from now