IRC channel logs
2024-01-26.log
back to list of logs
<gnu_srs1>youpi: Maybe you should apologise for being rude? If that exists in your vocabulary! <youpi>is asking questions being rude? <plasma41>At the risk of sticking my nose where it's not wanted, I've reread the backlog and I fail to see any cause for taking offense. <damo22>ive narrowed it down, either the address 0x7000 is no good for AMD FAM16h or the PIT aux2 does not work <youpi>what do you mean by 0x7000 ? <damo22>youpi: setting the AP boot code at memory address 0x7000 <damo22>i think 0x2000 is a better choice as the board has all zeroes from 0x2000 - 0x4000 <youpi>ok, I would be surprised that it'd be different from the intel value <youpi>pit, however, is something external to the x86 ISA, so I wouldn't be that surprised that it's not supported properly <youpi>0x2000 - 0x4000 being zero looks like only luck to me <youpi>it's the OS that decides what it puts where anyway <damo22>i changed it to 0x2000 and qemu rebooted! <damo22>but qemu has nothing there either <youpi>but I don't understand: are you trying to run an smp kernel ? <damo22>the AP_BOOT_ADDR is arbitrary, its something we chose <youpi>I though you didn't have the up kernel running so far <youpi>do we actually make sure that it's not allocated ?? <damo22>how do we choose a very low address that is not allocated? <youpi>start_other_cpus() could memcpy to whatever place <youpi>it seems it needs to be on a round page, that's not an issue at all <youpi>at the start_other_cpus() call time, we have plenty of allocation possibilities :) <youpi>ah, just one constraint: it has to be within 1MB <youpi>since the AP starts in real mode <youpi>but we can allocate that early right after the allocators are initialized <youpi>before various stuff allocate memory <youpi>and just check that it's indeed below 1MB <damo22>it needs to be at a physical address on a page boundary and below 1MB <youpi>page boundary is really no problem, vm_page_grab() etc. gives you that <damo22>apboot_addr = (void *)pmap_grab_page() ? <youpi>there's some more work in the apboot code but that's the idea <youpi>more work: get the apboot address with some jmp 1f ; 1: pop %eax trick, to replace the hardcoded AP_BOOT_ADDR value <damo22>smp_startup_cpu(apic_id, _kvtophys(apboot_addr)); <damo22>#define M(addr) (addr - apboot + apboot_addr) <damo22>hmm maybe if i store the pointer as a vm_offset_t <damo22>i dont know how to replace the AP_BOOT_ADDR value <damo22>i cant use +expr &apboot_addr APBOOT_ADDR <damo22>because that is the address of where the value is stored <damo22>can i dereference that pointer in asm? <damo22>is there a way to tell gnumach to avoid a page? <anatoly>what does "ap" in "apboot_addr" stand for <damo22>i cant remember, but it refers to all the processors that are not the base one <anatoly>is it some predefined globally available address few other cpus to start from when they're executed or something like this used during bootstrapping of the non-base cpus? <damo22>well you have to populate ram with some boot code below 1MB and send an IPI to wake up the processor with the address to start from <damo22>my problem is, the address is not easily accessible from asm because i allocated it in C code <damo22>but the startup cpuboot.S requires the address <damo22>since the cpu is not initated by a function call <damo22>i think it must be predetermined in the linker script <anatoly>so you won't need to send the address? <damo22>the cpu just wakes up in the address <anatoly>then I didn't read your first sentence right <damo22>the BSP sends an IPI to the AP with the address to start from <damo22>maybe i can read the current address in the AP startup code? <anatoly>heh, you confused me again by using "send" word :-) but, np, it's my missing knowedledge <damo22>how do i ljmp to a variable address? <anatoly>a search engine says push / ret or move eax / jmp eax <anatoly>and I'll go and hide in case I annoy you :-D <damo22>i dont understand how to do this, ljmp in real mode takes only an immediate <anatoly>I can't answer such questions. I can only google and suggest based on others suggestion. But I'm not sure weather it's useful for you, or you know how ot make such jump but don't know how to prep address for the jump from what you have :-) <anatoly>I'd asssume the later is the actual problem <damo22>/home/demo/git/gnumach/build/../i386/i386/cpuboot.S:115:(.text+0x227e3): relocation truncated to fit: R_386_16 against `.text' <anatoly>hey damo22 can you tell me if I'm correct in my assumption: we've got some address, we make sure we have linear/physical version of it if it's not, then find it's segment number and offset, then put this two numbers at some address and then do far jump by the value at the memory address <damo22>i dont know if BOOT_CS = 8 means we have decided to use 0x8000 ? <anatoly>If CS stands for code segment then it looks like a number if the segment <solid_black>youpi: what I meant there was that e.g. Guix people must already have the /lib/ld.so.1 -> /gnu/somewhere/ld.so.1 symlink (not only /lib/ld.so -> ...), otherwise glibc binaries wouldn't run, so we wouldn't be breaking them if we switched to ld.so.1 everywhere <youpi>damo22: that's the "get the apboot address with some jmp 1f; 1: pop%eax trick" part I meant <youpi>except it should have been "call jmp 1f; 1: pop %eax" <youpi>except it should have been "call 1f; 1: pop %eax" <youpi>call can use relative address <youpi>ah, wait, you don't have a stack yet <youpi>ok then you can revert to another idea I also had: just put it into a variable <youpi>and you can set its value from C before the memcpy <youpi>and read it from the asm code <youpi>ah, no, mem addresses can't be relative in mov instructions, I guess <youpi>another way then is to just modify the produces binary instructions, from the C code <youpi>i.e. mark the mov instruction with a label <youpi>and from knowing the produced binary bytes, offset from that into the address part of the mov instruction <youpi>so you can modify it from the C code <youpi>anatoly: at boot, processors are in real mode, not protected mode, so theres' no linear/physical question, it's all read addressing mode, i.e. physical <etno>youpi: >> offset from that into the address << a kind of linker relocation <youpi>damo22: BOOT_CS simply maps to the gdt_tmp array <youpi>.quad 0 skips segment 0, then you have segment 8 <youpi>etno: the linker cannot do it <youpi>it cannot know what address it will be at <youpi>but, yes, a kind of runtime linker relocation <etno>It would be elegant to be able to give the AP a stack pointer, where a return address would be stored, and then have it do RET. <etno> You must have already though about that <etno>Does the AP knows its "index number" ? <youpi>damo22: alternatively you can indeed tell gnumach to avoid a page by changing BIOSMEM_BASE <youpi>since e820 might very well actually tell you it's reserved <youpi>etno: you cannot set anythingto the AP <youpi>you *only* give it the ip address <youpi>the AP doesn't know anything else <youpi>it can determine its index number with cpuid yes, but then what :) <youpi>that doesn't give it the dynamic address you got <damo22>its very difficult to dynamically allocate it <youpi> « In legacy IA-32 architecture, addressing relative to the instruction pointer is available only with control-transfer instructions » <youpi>so yes, the relative addressing is only for jumps & calls <youpi>but since you don't yet have a stack yo u cannot do call <youpi>and jump doesn't let you ge the result <etno>youpi: last question: could the APs stacks be allocated statically? <youpi>etno: but you cannot really know what area could be available <youpi>e820 might tell you it's reserved <damo22>can we agree on a single page used just for real mode init, and block gnumach vm from allocating from that page? <etno>I am used to handle the bootstrap myself, and now I realize that with grub, we run already in extended memory... Ok <etno>If our code was in lower memory, that would be possible <etno>Gnumach is way too big for that :'-) <youpi>oh wait, since we are in real mode, we can use segmentation <youpi>see linux' arch/x86/realmode/rm/trampoline_32.S <youpi>you can actually access the variable in the same segment without knowing what segment that is <youpi>since it's just a 16bit address <etno>There is this unreal mode, is that what you suggest youpi? <youpi>it's just that in real mode your pointers are limited to 16bit <youpi>and when booting the AP you specify which one it is <youpi>so addresses are within that <youpi>so the code can just read the variable from that <damo22>arch/x86/realmode/rm/realmode.h:#define LJMPW_RM(to) .byte 0xea ; .word (to), real_mode_seg <damo22>and real_mode_seg is a special section in the linker script <damo22> * This must be open-coded since gas will choke on using a <damo22> * relocatable symbol for the segment portion. <youpi>mmm, I don't see why they do that, and why that'd be needed <youpi>you can just put the variable in the same segment, that'll just work <youpi>in the binary you'll probably have to put it in the .data section since you'll want to be able to write to it, but that's fine <youpi>you'll memcpy it anyway before letting the AP execute it <youpi>anyway that's not what I wanted to refer to <youpi>rather mov %cs , %eax ; mov %ax, %ds ; movl tr_start, %eax <youpi>where tr_start is just a label that is not far, and contains the address of the apboot code <youpi>(by making sure that the allocation indeed managed to allocate that low) <damo22>why cant i create a variable in asm like: <youpi>you can but it'll be at 0x1000sopmething <youpi>after the memcpy it'll be somewhere in the AP memory areay <youpi>and you don't know the absolute address <youpi>but you can know the 16bit offset since that is constant (that's the tr_start part of the movl) <youpi>and with the %cs -> %ds transfer, you'll know the real segment <youpi>with that, I really have to go <etno>sneek: later tell youpi: out of curiosity, is it possible to link gnumach and say that an elf segment of it has to be somewhere in the 1st megabyte ? <solid_black>I'm not sure if the resulting system counts as gnu or not <solid_black>it's built with gcc/binutils, uses gnumach + hurd + glibc <solid_black>but other than that, busybox & stuff; and GNU software is installable, but non-default <anatoly>I have no strong knowledge in this area <anatoly>GNU Hurd - tells it's a GNU's project called Hurd <anatoly>GNU/Linux - tells us it's kernel + userland <anatoly>Debian GNU/Hurd - in this case I don't what the names tells us <anatoly>Foobar GNU/Hurd - I guess is what fits in this examples <anatoly>I believe it's worth of nothing to spend time thinking about instead writing code, the time will show :-) <solid_black>should we use ldconfig or no? we don't have any paths beside /lib and /usr/lib (though, I guess, /usr/local/lib too?), but it also functions as a library cache for performance reasons <anatoly>solid_black: have you forked busybox? :-) maybe weetbix? :-D <anatoly>the typo you did on the screenshot - busybix <solid_black>installing packages kind of works, but results in lots of errors because of missing ldconfig, so let me rebuild with that and see how it fares <anatoly>youpi: i know cpu starts in real mode and what it has is 20bit address in the end but you said no segment and offset and it's from protected but then later you use these terms in the context of real mode and the link says the same. This is what I meant by these terms. Not what it means in protected mode :-) <anatoly>solid_black: I did more progress with container file, so it boostraps from the plain alpine container, builds patched abuild anf then uses it on the next stage for bootstrapping cross-compiler, etc. Then the artefacts of this will be used to create final container with cross-compiler installed. This we can share for pepople to quickly cross build the rest on their linux machine. Then it can be used further <anatoly>All of that can be moved to CI for automation <anatoly>and extended (following alpoine ci example) to build whole distro in the end <anatoly>Not yet, I haven't finished testing build cross-compiler yet, it failed and this steps takes a lot of time :-) <solid_black>ugh does busybox really list all the signals it knows about <solid_black>and SIGLOST is not among them, which explains the error <anatoly>And I need to add another step to move cross-compile + some base package to the final image <anatoly>And I need to add another step to move cross-compile + some base packages to the final image <anatoly>Or maybe move only cross-compile packages. I'm not sure about it and it's not important for now <damo22>ok now my problem is the pmap_grab_page gets a physical address at physical apboot_addr = 0x75db9000 <damo22>how do i allocate a low address? <anatoly>I was reading youpi answers but didn't get the thing is that you don't need to know segmen but only offset in it because the segment is the same <damo22>i fixed that specific issue with the boot process, but my problem is i cant seem to allocate a page of memory below 1MB <janneke>solid_black: fwiw, guix does not have any ld.so symlinks or shared location <janneke>every binary has the exact location if its dynamic linker built in <janneke>dependencies are strict in guix (and nix), never soft or shared <solid_black>the system booted with only gnumach, hurd, alpine-base installed, the rest I 'apk install'ed at runtime <anatoly>damo22: I'm curious what do you need to allocate a page of memory below 1MB <solid_black>janneke: so how do you build binaries like that, do you pass LDFLAGS=-Wl,-dynamic-linker /exact-path/ld.so.1? <damo22>anatoly: apboot_addr = phystokv(vm_page_to_pa(vm_page_grab_contig(PAGE_SIZE, VM_PAGE_SEL_DMA))); <solid_black>surely rpath is not the same thing as ELF interpreter path <jpoiret>yes, I just woke up mb, I think we have both <solid_black>anyways, so we wouldn't break anything for you guys if we were to change upstream gcc to emit /lib/ld.so.1 by default? <janneke>so, no worries, but it's nice people are thinking of us :) <damo22>hmm doesnt the code sitting at apboot symbol become cs:0 on the AP? <janneke>(as long as the (basename) of the linker remains the same) <damo22>or does it become cs:X where X is the low address of the transfer <damo22>hmm why cant i just use page 0 of RAM? <solid_black>does mkdir("existing") on a r/o fs return EROFS and not EEXIST? <solid_black>yes, if I patch libdiskfs to return EEXIST first, then OpenRC proceeds further <gnucode>hmmm, I am slowing creating a website via org-mode. <youpi>Pellescours: they are, mach avoids using the first 64KiB <sneek>Welcome back youpi, you have 1 message! <sneek>youpi, etno says: out of curiosity, is it possible to link gnumach and say that an elf segment of it has to be somewhere in the 1st megabyte ? <youpi>well, elf allows to do so, but grub will frown upon that <youpi>and the thing is: we *don't* need that <etno>youpi: ok, thanks for the info 👍 <youpi>alloc+memcpy is all we need to do <youpi>and cope with the relocation, which should really not be a problem, all OSes have the same question <Guest86>sorry I meant to send that it myself <gnucode>sneek later tell solid_black You just need to clone the repo, install emacs, run ./render-locally, and you should be able to generate the website. <damo22>youpi: i set the data seg the same as the code seg, but i cant seem to locate the offset, because the symbols are resolved at compile time, not after they are moved <damo22>ljmp requires an immediate, which is absolute offset <damo22>so how do i refer to a symbol that will have a different offset after it is moved? <damo22>damn i am getting /home/demo/git/gnumach/build/../i386/i386/cpuboot.S:115:(.text+0x22806): relocation truncated to fit: R_386_16 against `.text' <damo22>when i try to define the trampoline_start symbol <damo22>movl tr_start, %eax in 16 bit mode only stores the bottom 16 bits of the address <damo22>so how do you store a long address to a register? <damo22>addr32 movl trampoline_start, %eax