IRC channel logs
2023-10-06.log
back to list of logs
<damo22>nice thing about this hw is its fanless <almuhs>damo22: i had problems with rumpdisk in many Thinkpad models. In the T410, if i remember well, the installer in rumpdisk mode didn't detect the harddisk. Similar in T60 and other models. In some cases, the installer detected the harddisk but not the CD drive <almuhs>so it's possible that rumpdisk has any issue in physical hardisk <youpi>damo22: perhaps it's the access to the hdd which is slow then <damo22>youpi: it is using /dev/sda which is a dedicated msata SSD <damo22>i think its booting with rumpdisk too <youpi>ok but I mean possibly the disk driver itself gets slow for whatever reason <youpi>right now I'm seeing that using the qemu ahci driver is quite faster than letting qemu pick up its default driver <youpi>kvm -device ahci, ... -drive file=... -device ide-hd,... -> fast <damo22>qemu-system-i386 -M q35,accel=kvm -m 4096 -net user,hostfwd=tcp::8888-:22 -net nic -curses -hda /dev/sda <damo22>shall i shut it down and retry with different params now? <youpi>with q35, I do get a fast speed <damo22>yes i always run with that param, the -M pc is slow as <youpi>I don't see a good reason why it should be that slow <almuhs>during the hurd booting in smp mode, when the boot freeze, my fan speed up, like the cpu make a big effort. This usually happens when a process enter in a infinite loop. Could gnumach are entering in a infinite loop in some scenarios? <damo22>yes its not ready, something is broken <damo22>youpi: i cant use -cpu host or hurd hangs <damo22>however i have switched off rumpdisk for now <damo22>perhaps it will be faster with gnumach ahci <damo22>almuhs: yes i have seen that error, its probably because an interrupt is lost or something <youpi>such hang is not supposed to happen, it's probably worth investigating <almuhs>anyway, i only got a shell with smp from time to time. Around 99% attempts fails the booting in smp mode, hanging during the boot <damo22>almuhs: did you use my experimental patch? <damo22>it makes most of the hangs go away <almuhs>but that is a ugly patch, it's not? <almuhs>this patch don't solve the origin of the problem <damo22>well its not too bad, it just makes more threads go on the run queues <damo22>instead of dispatching direct to idle processor <damo22>in theory it should be slower yes, but for some reason its faster <almuhs>but the speed is not as important problem than the hangs <damo22>yes most of the hangs are also gone with this patch <almuhs>"../include/mach/std_types.defs", line 32: overriding previous definition of char <almuhs>oh! "configure: WARNING: unrecognized options: --enable-apic, --enable-ncpus, --disable-linux-groups" <almuhs>i'm trying to compile from your repository's sources <damo22>my sources are master + one commit <damo22>you may need to check out "fixes" branch <almuhs>testing with -smp 4 , hangs just before ask the username <damo22>yes it does hang randomly but works more often than without the patch <almuhs>ok, now i got successful login with -smp 4 <almuhs>what command can i use to check what processor executes each thread? <damo22>idk but you can install stress and run stress -c 4 <almuhs>all threads seems to runs in cpu 0 <damo22>i doubt it, there is 100% load on 4 separate processes <almuhs>how can i show all process inclusing the system process? <youpi>reporting this is possibly just not even supported <almuhs>we added this support 3 years ago <almuhs>in gnumach and hurd. I added a new field in the scheduler struct to add the last_thread, and i patched two hurd servers to recover this new field and fill the process table with the information <youpi>but is that where top reads it? <almuhs>yes, but they can take the info from similar sources, it's not? <youpi>ok, so that should be working <almuhs>how can i check the last processor using ps? <youpi>possibly it's not even implemented <almuhs>in that time, you told me a ps arguments to check it <youpi>but I just cannot remember such thing by heart <almuhs>ps has a TH column, but in blank <almuhs>ok, it seems that ps command in hurd is more limited than in linux <damo22>#ifdef HAVE_STRUCT_THREAD_SCHED_INFO_LAST_PROCESSOR <damo22>i dont see that defined anywhere in hurd <damo22>AC_CHECK_MEMBERS([struct thread_sched_info.last_processor] does this do some magic? <damo22>When member belongs to aggregate, define HAVE_aggregate_member (in all capitals, with spaces and dots replaced by underscores) <damo22> integer_t depress_priority; /* priority depressed from */ <damo22> integer_t last_processor; /* last processor used by the thread */ <damo22>#define THREAD_SCHED_INFO_COUNT \ <damo22> (sizeof(thread_sched_info_data_t) / sizeof(natural_t)) <damo22>is natural_t equivalent to integer_t ? <damo22>mach/i386/vm_types.h:typedef int integer_t; <damo22>mach/i386/vm_types.h:typedef unsigned int natural_t; <damo22>i guess theyre the same size at least <damo22>+ * Strictly speaking, this is wrong. The PIRQ should be routed <damo22>+ * to *both* the I/O APIC and the PIC, on different pins. The <damo22>+ * I/O APIC has a fixed mapping to IRQ16-23, while the PIC is <damo22>+ * routed according to the PIRQx_ROUT configuration. But QEMU <damo22>+ * doesn't (yet) cope with the concept of pin numbers differing <damo22>+ * between PIC and I/O APIC, and neither does the in-kernel KVM <damo22>+ * irqchip support. So we route to the I/O APIC *only* if the <damo22>+ * routing to the PIC is disabled in the PIRQx_ROUT settings. <damo22>+ * This seems to work even if we boot a Linux guest with 'noapic' <damo22>+ * to make it use the legacy PIC, and then kexec directly into a <damo22>+ * new kernel which uses the I/O APIC. The new kernel explicitly <damo22>+ * disables the PIRQ routing even though it doesn't need to care. <damo22>maybe that could explain some strangeness with interrupts? <damo22>qemu @ 886e0a5f31bf3d40dd8d9199674a4bad64942fde <solid_black>but as I understood youpi, we want to use PipeWire, like GNU/Linux distros do nowdays <damo22>pipewire is a stack that sits on top of ALSA <solid_black>so either we should support alsa, or PipeWire would have to gain a Hurd-specific backend <damo22>i think we should expose a JACK api <damo22>pipewire can sit on top of that if needde <Arsen>I don't think that makes sense <Arsen>JACK also has some of the flas noted in that email AFAIK <damo22>yes, users dont need to open() read() write() to a sound card <damo22>we need to expose a callback mechanism <damo22>that is the problem with linux sound <damo22>if JACK api is missing some things, we can extend its api <solid_black>so that linked wiki page looks like a list of things that people would like to improve in a potential alsa successor <Arsen>that doesn't require implementing jack, something more fit to mach seems to make sense <damo22>reason: i dont want to break all audio apps <damo22>we need to expose an existing api <damo22>but the internals can be better designed <solid_black>I thought it only implemented jack compatibility for clients <solid_black>keeping up with the trend, should we schedule a call with Wim Taymans? :D <damo22>he thinks polling a fd for IO is not bad for audio <solid_black>is there an intro to the jack api that you would recommend? <damo22>basically, you open a connection to a jack server, then you register some ports, and register a callback that you will be using to fill the buffers with, and start the server.... jack decides WHEN to call your callback to retrieve the audio data <solid_black>we want to reuse some existing drivers for sound cards, don't we? rump? <damo22>i was thinking to reuse ALSA low level drivers <damo22>they are a bunch of function pointers <Gooberpatrol66>what's the difference between the way jack works and pipewire works <solid_black>have a soundcard device driver run in its own translator (naturally) <Gooberpatrol66>i remember when i used jack i could only use 1 soundcard at a time and pipewire allegedly fixed that <damo22>yes, jack supports one sound card at a time <solid_black>and communicate with the PW deamon using a bespoke Mach IPC protocol (sound.defs perhaps) that is neither alsa or jack <solid_black>and then the apps/clients speak PA, or PW's natuve protocol, or Jack (or alsa?) to pw <solid_black>but its protocol is what most clients speak, to the PW daemon, or rather the pw-pa bridge <Gooberpatrol66>would the drivers use pipewire code to speak jack? (because pipewire implements jack API)? <damo22>i was thinking we could write a driver that exposes a filesystem node like /dev/snd that has some kind of API that can be implemented as a jack backend <solid_black>(also I cannot speak to whether pa is garbage, but Lennart is one of the smartest, most talented people I know about, despite all of the shit that gets said about him) <damo22>the main flaw with ALSA is that transfer of samples occurs by arrival of audio IRQ <damo22>the driver needs to be rewritten <solid_black>so is it true that jack also has a protocol that clients speak to a central daemon? and that daemon uses alsa to talk to the device? <solid_black>then I don't understand how jack is going to be involved in your plan <damo22>so applications dont need to changre <damo22>they can continue connecting to jack <solid_black>you want individual clients/apps to use the jack api/protocol? <damo22>it can sit on top if you really need something non-jack <solid_black>or do you want PW to be there only for providing PA compatibility for those clients that don't speak jack natively? <solid_black>and you're saying that pw already has a jack backend <solid_black>how would the jack daemon talk to the device in your plan? <damo22>open /dev/snd and use RPCs to talk a hurdish api that implements all the things jack needs <damo22>yes we would need to implement a hurd jack backend for jack <solid_black>and if someone wouldn't want to use Jack, they could also implement a backend for the same Hurdish protocol into pw? <Arsen>if I follow, you mean to implement not jack, but a backend for Jack to use, and then for pw to use indirectly via jack? <Arsen>why not just skip the middle man <Arsen>I like jack but I don't follow the design here <damo22>you can also do that Arsen if you wish <damo22>pw can run on top of jack already <damo22>so we dont need to do any work for that path <Arsen>... yes, but jack can't run on top of Hurd (yet) <solid_black>it would really help if you said "jack daemon" and not just "jack" (if the daemon is what you mean) <Arsen>so if the goal is running pw, porting jack rather than pw doesn't seem to be productive <Arsen>I don't, above someone said that pw implements a jack based backend <Arsen>as in, jack provides the audio devices <solid_black>pw, as far as I've heard, implements the jack client library <damo22>alsa -> jack server -> pw -> speakers <Arsen>it does that indeed, but apparently it also can be a jack client <solid_black>which then speaks the pw native api to the pw deamon, but this i'm not sure about <damo22>or alsa -> jack -> pw -> speakers <damo22>alsa -> pw -> jack api -> jack client <solid_black>can we have /dev/snd both support an alsa api (which is read/write/ioctl on the device node, right?), and the additional Hurdish API? <damo22>hurd sound -> jack api -> jack client (or pipewire -> pw client) <damo22>and one day someone can do hurd sound -> pw api <solid_black>the point of exporting the alsa api is that one could use jack or pw daemons even before we'd implement Hurd-specific backends into them <damo22>the whole point of rewriting alsa is to fix its flaws in its api <damo22>for example, i want to add port names <Gooberpatrol66>pipewire is designed to transfer data between containers, which could be important for hurd, given how compartmentalized it is <damo22>ALSA api does not export the port names of each i/o <damo22>also the device enumeration is not unique in alsa <damo22>so you unplug a sound card and plug it again and you might get a different card number <damo22>ALSA also supports OSS emulation which needs to die <Gooberpatrol66>so would a sound card driver emulate a jack daemon and then speak to another jack daemon that acts as the sound server? in your idea? <damo22>no, a sound card driver would expose a set of RPCs that something like a jack daemon would need <damo22>it should be a trivial port if the api matches roughly <damo22>the hard part is making the driver <damo22>so jackd would open(/dev/snd) and call rpcs that it needs to control it <damo22>the audio data does not flow through /dev/snd <damo22>it just registers a callback with a pointer to some buffer <solid_black>what sort of callback and a buffer, how would that map to the Mach RPCs? <solid_black>would it not still be a Mach message carrying the buffer contents? <damo22>i dont think we can pass audio data as a mach message, wont it be too slow? <solid_black>or, would jackd mmap the device buffer directly? (is that possible?) <damo22>yes, shared memory to a small ringbuffer? <damo22>the audio driver would mmap the device buffer directly and somehow we need to share that with the caller <damo22>no the audio driver would start and stop the card <damo22>we just need to fill its buffer somehow <damo22>the audio driver would know when to do this <damo22>scheduling a writer and a reader probably <damo22>the arrival of the audio IRQ tells you when the hw pointer is at the end of the cycle <damo22>but you dont just copy the buffer when this happens <damo22>you use the timestamps of arrival of the audio IRQs to calibrate a delay locked loop <damo22>then you can ask the question : where is the buffer pointer now <damo22>at any time you want in between the audio irq <damo22>and use some kind of timer interrupt to schedule the transfer when you want <damo22>depending on the requested latency <damo22>this all happens in the audio driver <damo22>but its controlled by jackd, and the audio driver will call the callback provided by the user application <damo22>yeh i havent figured that part out yet <damo22>the audio driver basically writes and reads zeroes until you fill it with some audio data <damo22>and if the callback blocks for too long, and cant deliver the audio in time for the next cycle, the audio driver drops the frame and you get silence <damo22>unless the audio driver is in the same process as jackd <damo22>somehow jackd gets access to a mmap'd buffer <damo22>jackd handles all the routing between different audio applications <damo22>cant one process call an RPC that mmaps the same buffer as another process and return a pointer to it? <damo22>i dont know how shared memory works <damo22>but user audio basically takes a buffer of floats and processes them into another buffer of floats <damo22>solid_black: jackd solved that problem somehow already, because jackd supports multiple processes connecting to it and registering callbacks with jackd so jackd can call them when it needs to fill the buffers <solid_black>Wayland itself, the protocol and the supporting libraries, might have been ported already <azert>I was reading damo22 plans for audio on the logs <solid_black>the clients (such as gtk) should be buildable with wayland support, should the distros ship wayland <solid_black>I've ported Owl, but Owl uses GNUstep as its backend, and that relies on X11 again <solid_black>this is cool and all, but it's not a solution for graphics on the Hurd <solid_black>Weston, the original reference Wayland compositor, has removed its framebuffer backend in 2022 <solid_black>is there any chance we could get that? what does rump do? <Arsen>fwiw managarm runs modern weston <Arsen>so getting enough DRM to run isn't impossible <azert>Maybe one can use a rump drm implementation? <damo22>we should probably port it if not <damo22>because netbsd drm is based on linux drm but is somewhat borked <damo22>i was working on netbsd's audio driver to introduce a DLL to provide more accurate buffer pointers <azert>Regarding audio, one could maybe write drivers as Jack clients? <azert>Not very hurdish, since it won’t be expose to the file system, but is there a reason not to? <damo22>jackd does not know how to drive cards directly <azert>But the jack api works for both input as well as output, right? <azert>So if the client knows how to drive a card, why wouldn’t it work? <damo22>its not designed to work that way <damo22>jackd cant read or write to a card it doesnt have a backend for <azert>It would just read/write to a client <azert>That happens to be an userspace driver <azert>But maybe I just don’t know jack well enough <damo22>thats a clever idea, but inverts the concept of a playback port <damo22>and exposes all the ports that the card supports <damo22>the card would show up as a client <damo22>but the process graph probably would not function correctly <damo22>because jackd would try to write to the backend <damo22>i want to reuse ALSA low level functions to deal with the card specifics <damo22>but implement the streaming from scratch <azert>I think you can connect clients to clients directly already, excluding the backend from the picture <azert>The backend would just not be there <damo22>yes but the process graph is computed from the root <damo22>i dont think it would operate as expected <damo22>i think the whole point of the graph is to minimise context switches between threads implementing each client's audio processor <damo22>the client for the card would not actually process anything, it generates samples and consumes samples <damo22>but can it be a leaf and a client ? <azert>I don’t see why not but I’d ask Jack developers <damo22>actually i implemented this already for QubesOS <damo22>i connected jack as a client to another jack instance on dom0 <azert>solid_black: I’ve looked at sys/external/bsd/drm2 in netbsd tree. They just ported the Linux drm, if they can do that probably it means it is portable? <solid_black>that it's portable in theory we know, it's also been ported onto FreeBSD IIRC <solid_black>the question is, will we get it "for free" with rump, or would we also have to invest into porting it <azert>How hard is it to create a new rump module? <youpi>Gooberpatrol_66: yes but there are missing changes in a few repos before it can be built at all <youpi>some of them are waiting for a CI to be integrated, which means waiting for libc to be released <youpi>hopefully that shouldn't take too many weeks from now 
