IRC channel logs

2023-09-23.log

back to list of logs

<almuhs>i'm learning to compile hurd, excuse me the delay
<almuhs>i'm compiling upstream's ext2fs from my hurd vm
<almuhs>ok, it compiled fine, but without your hack. Now i will try to add the hack
<azert>did you check that without the hack it gets stuck?
<almuhs>no
<almuhs>not with this upstream code
<azert>the hack doesn't do anything if gnumach is not compiled for smp
<almuhs>yes, i know
<azert>ok
<almuhs>but first at all i have to locate a good place to put the snippet
<gnucode>azert: that's kind of cool! I didn't realize that ext2fs has dataraces. Doesn't libdiskfs have similiar problems?
<azert>gnucode who knows
<gnucode>very cool.
<almuhs>before apply the hack, i go to upgrade my vm and my gnumach
<almuhs>where can i copy the snippet inside ext2fs code?
<azert>I'd put it just in the main ?
<azert>before everything
<almuhs>where is the ext2fs's main?
<almuhs>ok, in ext2fs.c
<gnucode># dpkg-reconfigure keyboard-configuration
<gnucode>they have Pinephone Keyboard on this list!
<almuhs>make[1]: *** No hay ninguna regla para construir el objetivo '/usr/include/i386-gnu/bits/auxv.h', necesario para 'wire.o'. Alto.
<azert>what is it?
<almuhs>the error when i execute "make ext2fs"
<azert>just type make for the whole hurd?
<azert>it takes a few minutes
<azert>i'm not sure individual server can be compiled
<azert>last time i tried i got a similar error
<almuhs>ok, removing build directory and creating an empty new directory works
<almuhs>compiling now
<almuhs>(and now you know that i am spanish ;-))
<almuhs>ext2fs compiled
<almuhs>oops... i have execute "make install", and now i think that all translators are compiling
<gnucode>almuhs: cee. :)
<azert>you can just copy the ext2fs executable perhaps
<almuhs>yes, i did it
<almuhs>but i am afraid to stop the make install
<almuhs>make install finished. crossing fingers before reboot
<almuhs>ok, now i have not network
<azert>does it boot ext2fs?
<almuhs>wait
<almuhs>at moment keeps freezed in auth server
<gnucode>bummer.
<azert>ah
<almuhs>what is bummer?
<azert>he was expecting the thing to go to login
<almuhs>but most of my translators are broken
<azert>do you mean they all have races?
<azert>fine, gotta go to bed, thnx for the effort!
<almuhs>ok, i'm fixing my hurd
<azert>i read backlogs if you have news!
<almuhs>reinstalling vm
<gnucode>almuhs: I thought that "at the moment freezed in auth server" meant that the "hack" to force ext2fs to only run on one cpu failed.
<almuhs>maybe
<almuhs>but i want to check in a healthy hurd
<gnucode>sounds good.
<gnucode>almuhs: what is a healthy hurd?
<almuhs>a hurd without buggy translators
<almuhs>the make install replaced all hurd translators by upstream's versions, which lacks some patches to work properly
<damo22>yeah dont run make install
<damo22>just run make, and manually copy the translator you are testing to your system
<almuhs>waiting to compile
<almuhs>corrupted filesystem, i don't get fsck
<gnucode>almuhs: are you running qemu?
<almuhs>yes
<damo22>e2fsck the filesystem from linux
<almuhs>i'm trying it, but i don't get
<almuhs>fsck doesn't allow to fix a mounted system
<damo22>yeah you have to unmount it
<almuhs>but then... how can i access it?
<damo22>you dont fsck a running filesystem
<almuhs>yes, but i don't know how to get a device
<damo22>where is your hard disk image
<damo22>is it an image or a device
<almuhs>hurd_qemu/hurd.img
<almuhs>it's a image
<gnucode>almuhs: I think the easiest way to fsck a qemu filesystem, is to use loseup I think.
<gnucode>You can then use your linux box, to fsck the qemu image.
<gnucode>then start the hurd like normal.
<damo22>ok so, losetup -P option
<almuhs>what option?
<almuhs>what is the syntax?
<damo22>i cant remember the syntax
<damo22>man page
<gnucode>OR you can try booting up qemu...if it boots to a login prompt, then you can login via root
<gnucode>umount /home && umount /
<damo22>you cant umount /
<gnucode>damo22: I have.
<gnucode>in real hardware.
<damo22>no it doesnt umount root it sets it ro
<almuhs>in qemu i had many problems, fsck -y didn't worked in emergency console
<damo22>write a script in linux that sets up loopback mount and creates /dev/loop0pX
<almuhs>it says "failed to parse fstab"
<damo22>then you can e2fsck -y /dev/loop0pX
<gnucode>damo22: I am not disagreeing with you, but I promise you that I have logged in as root, and typed in "# umount /home && umount / && fsck.ext2 /dev/sd0s1 && /dev/sda0s6 && reboot. That fixed my issues. :)
<almuhs>damo22: i have this script, but i can't fsck a mounted filesystem
<gnucode>almuhs: give me a second...
<damo22>it has to be unmounted first
<damo22>or not even running
<gnucode>I know that I know how to use lose2up. I think I have a script somewhere...
<almuhs>but if i unmount the filesystem, the device also disappears
<gnucode>almuhs: cat /etc/fstab
<damo22>yes, that is what losetup is for
<damo22>losetup sets up a loopback block device node
<damo22>it should persist even if the virtual device is unmounted
<almuhs># /etc/fstab: static file system information.
<almuhs>#
<almuhs># <file system> <mount point> <type> <options> <dump> <pass>
<almuhs>/dev/wd0s1 / ext2 defaults 0 1
<almuhs>/dev/wd0s5 none swap sw 0 0
<almuhs>/dev/cd0 /media/cdrom0 iso9660 noauto 0 0
<almuhs>gnucode: my fstab
<almuhs>as curious, from linux i can mount the filesystem without error
<damo22>you probably need to set the dump/pass options differently so it auto fscks
<gnucode>yes. change /dev/wd0s1 <dump> option to 1
<gnucode>I think.
<almuhs>ok
<damo22>interesting, i have no fstab
<gnucode>almuhs: I think this following should work:
<gnucode>in qemu, boot up the Hurd.
<gnucode>at the login prompt login as root.
<gnucode>umount /
<almuhs>setting dump to 1 didn't worked
<almuhs>gnucode: the problem is i can't access to login
<gnucode>fsck.ext2 /dev/wd0s1
<gnucode>ok.
<gnucode>almuhs: then the best way is to use losetup
<almuhs>because during the booting i have an error which force me to make fsck
<gnucode>We should create a script for Linux "start-hurd.sh"
<damo22>you should power off your qemu to fsck from linux
<almuhs>yes
<gnucode>when you run that command it auto fsck's all of the Hurd's filesystems, then starts qemu.
<damo22>then use losetup -P to scan the image for partitions and it exposes the device
<gnucode>let me see if I figure out the losetup syntax.
<damo22>then you can e2fsck the virtual block device
<almuhs> https://pasteboard.co/f4sox2yTD77o.png
<almuhs>this is the problem
<damo22>you must have a syntax error in fstab
<almuhs>losetup -P hurd.img says that it can't find the device
<gnucode>ok almuhs I got you.
<gnucode>just a second
<damo22>losetup -P -f /dev/loop0 hurd.img
<damo22>probably as root
<gnucode> https://paste.debian.net/1292865/
<almuhs>ok, i will try it
<damo22>just use -P as i suggested, you dont need to compute offsets manually
<damo22>it reads the partition table and autodetects the partitions
<almuhs>almu@debian-t480s:~$ sudo losetup -P -f /dev/loop0 hurd_qemu/hurd.img
<almuhs>losetup: argumentos inesperados
<almuhs>unexpected arguments
<damo22>hmm
<almuhs>ok, without -f
<almuhs>now i have loop0
<damo22>you should have /dev/loop0pX
<almuhs>yes
<damo22>where X is the partition number of your /
<damo22>is it 1?
<almuhs>almu@debian-t480s:~$ lsblk
<almuhs>NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
<almuhs>loop0 7:0 0 10G 0 loop
<almuhs>├─loop0p1 259:4 0 9G 0 part
<almuhs>├─loop0p2 259:5 0 1K 0 part
<almuhs>└─loop0p5 259:6 0 975M 0 part
<damo22>e2fsck -y /dev/loop0p1
<gnucode>it should be loop0p1 right? That's where / was in your fstab
<almuhs>yes
<almuhs>i need to calculate offset
<damo22>what offset??
<damo22>its there
<almuhs>The superblock could not be read or does not describe a valid ext2/ext3/ext4
<almuhs>filesystem. If the device is valid and it really contains an ext2/ext3/ext4
<almuhs>filesystem (and not swap or ufs or something else), then the superblock
<almuhs>is corrupt, and you might try running e2fsck with an alternate superblock:
<almuhs> e2fsck -b 8193 <device>
<almuhs> or
<almuhs> e2fsck -b 32768 <device>
<damo22>what param did you do
<damo22>dont pass /dev/loop0
<almuhs>ok, i was setting the disk instead partitiion
<almuhs>sudo e2fsck -y /dev/loop0p1 worked
<damo22>yes
<almuhs>i go to try qemu
<damo22>you probably need to disable the loop first
<damo22>sudo losetup -D
<almuhs>it worked without disable, but i go to shutdown and disable
<almuhs>now disabled
<almuhs>i think that i will add a new script to my collection
<damo22>;)
<almuhs>done https://github.com/AlmuHS/gnumach_dev_scripts/blob/main/fix_filesystem.sh
<damo22>sudo losetup -D
<damo22>at the end
<almuhs>true
<almuhs>i'm fixing some conflicts and then i will push it
<almuhs>fixed
<gnucode>when I do a wget on https://github.com/AlmuHS/gnumach_dev_scripts/blob/main/fix_filesystem.sh, I get a weird file as the result...
<almuhs>gnucode: you must press raw in the browser. I pressed it by you https://raw.githubusercontent.com/AlmuHS/gnumach_dev_scripts/main/fix_filesystem.sh
<gnucode> thanks.
<gnucode>I am using netsurf at the moment.
<almuhs>when i repair my virtualbox installation, i will try to use the DE over Hurd again. Qemu with DE runs buggy
<gnucode>almuhs: I am getting really happy running it in real hardware. super fast.
<gnucode>though I understand that running it in qemu or Xen makes debbugging it much easier.
<gnucode>for serious development. Also I am using i3 on the Hurd. Very lightweight.
<damo22>i reboot my vm every 10 minutes or so
<gnucode>damo22: why?
<damo22>to test a kernel change
<almuhs>XD
<gnucode>damo22: that's a pretty rapidly fast kernel cycle.
<damo22>i have most of it compiled, and tweak something, then install the kernel and reboot
<gnucode>damo22: are you running Debian Hurd? probably 64 bit?
<damo22>debian hurd 32 bit
<gnucode>gotcha.
<damo22>i usually ssh into the vm to develop inside hurd
<skeletonlamb>any good places to get started with Hurd?
<damo22>good places?
<almuhs> https://www.debian.org/ports/hurd/hurd-install
<skeletonlamb>almuhs: Thank you.
<almuhs>and my guide. Some things are obsolete, but it's very graphical https://gist.github.com/AlmuHS/f0c036631881756e817504d28217a910
<skeletonlamb>Thanks 2x. I'm going to try to install it via kvm.
<almuhs>i recomends qemu
<gnucode>thanks! I need to put that information on the wiki!
<almuhs>kvm via qemu is easier than only kvm
<skeletonlamb>I'm using qemu.
<almuhs>this is my qemu script https://github.com/AlmuHS/gnumach_dev_scripts/blob/master/qemu-hurd.sh
<damo22>bbl
<gnucode>almuhs: you should add at the top of your qemu-hurd.sh script your losetup. That way your filesystem will always be checked.
<almuhs>it's a good idea
<gnucode>gotta get going soon.
<almuhs>last commit before go sleep. 3:48 AM in spain
<skeletonlamb>Just got it running. Thank you.
<gnucode>10pm in the U.S.
<gnucode>night all.
<skeletonlamb>though one question, how would I ssh in? I have added the flags as adviced by debian, however when running ssh root@127.0.0.1 -p 2222 or ssh localhost -p 2222 it fails to handle the password.
<damo22>skeletonlamb: i think sshd is configured to reject root logins by default
<damo22>you might need to tweak /etc/ssh/sshd_config
<damo22>or login as demo
<damo22>non-root user
<damo22>skeletonlamb: how did you go?
<damo22>youpi: how do i set a breakpoint before segmentation is set up?
<damo22>i tried "(gdb) b *0x10040c0
<damo22>it hits a trap_from_kernel really early
<damo22>s/segmentation/paginhg
<damo22>for example if i set a breakpoint on c_boot_entry, it just fails to insert the breakpoint because it cant access 0xc1000000
<damo22>i cant set a breakpoint early
<damo22>because when it jumps to protected mode, the memory segmentation is different, so the breakpoint it inserted is at the wrong addres
<damo22>gah
<damo22>-smp 1 is stable at least now
<damo22>its about 2x slower
<damo22>got it
<damo22>i dont think CPU_NUMBER can get much more optimised:
<damo22>+#define CPU_NUMBER(reg) \
<damo22>+ movl MY(CPU_ID), reg;
<damo22>goodnight
<almuhs>damo22: in a boot i got login screen, but freezed just before ask user. But now i don't get repeat this
<almuhs>NCPUS=4, -smp 2
<almuhs>i think that we need to apply the same hack to libdiskfs
<almuhs>to force that libdiskfs use a only processor
<gnucode>hello friends!
<almuhs>there are so many races condition. Depending of the moment, sometimes the boot freeze after load exec, others after load startup, other times after INIT... and in a few reach "cleaning temporary files"
<gnucode>fixing race conditions seems hard...
<almuhs>a good start could be to implement the emply codeblocks in pcb.c and other files
<almuhs>in stack_handoff() and switch_context()
<almuhs>maybe the switch_context could be the most relevant
<almuhs>#if NCPUS>1
<almuhs>#warning SMP support missing (avoid races with io_perm_modify).
<almuhs>and i386_io_perm_modify() in io_perm.c
<youpi>that's only about I/O permissions, that won't be a problem
<almuhs>almu@debian-t480s:~/gnumach$ grep -rn '#warning SMP support missing'
<almuhs>i386/i386/pcb.c:295:#warning SMP support missing (avoid races with io_perm_modify).
<almuhs>i386/i386/pcb.c:369:#warning SMP support missing (avoid races with io_perm_modify).
<almuhs>i386/i386/io_perm.c:314:#warning SMP support missing (notify all CPUs running threads in that of the I/O bitmap change).
<youpi>processes make very few calls to io/perm functions
<almuhs>but we have many problems with ext2fs
<youpi>ext2fs doesn't use i/o perm
<almuhs>and libdiskfs?
<youpi>I'm talking about the process
<youpi>only actual device drivers use i/o perms
<almuhs>yes, but don't ext2fs make calls to devices?
<youpi>not inside its own process
<youpi>it makes an RPC
<youpi>so it'll be another process
<almuhs>maybe the problem is in the another process
<almuhs>in the process which manage the hard disk
<youpi>still
<youpi>again, such calls are seldom
<youpi>put a printf there is you want to see
<almuhs>what is seldom?
<youpi>io_perm_modify calls
<youpi>drivers basically just call it at initialiszation and are done with it
<almuhs>yes
<almuhs>i have to check it. because most of freezes are during the access to hard disk
<almuhs>during filesystem checking, during swap enabling, during cleaning of temporary files...
<youpi>if it's a race within ext2fs, that's to be expected, yes
<almuhs>but swap not is ext2fs, is it not?
<almuhs>maybe a race in rumpdisk?
<youpi>could be
<youpi>I'd say try to boot off an initrd, to get rump off the picture
<almuhs>how can i do it?
<youpi>you can see on debian's .iso image , boot/grub/grub.cfg
<youpi>the boot_one function
<youpi>you'll need to ramdisk patch from the debian patch
<almuhs>ok, i will try to find it
<almuhs>i've found the boot_one function in grub.cfg https://paste.debian.net/1292924/