IRC channel logs

<Pellescours>It also says that each CPU should have an unique SMBASE, I don’t know if we reallocate SMBASE for APs

<damo22>Pellescours: i think its simpler than that, the interrupt stacks are probably not configured for APs

<damo22>i mean, i set them in %esp but im not sure if that is enough

<Pellescours>I’m reading the doc to try to understand how AP should be started, I’m also looking at linux code

<Pellescours>damo22: https://github.com/apple/darwin-xnu/blob/main/osfmk/i386/seg.h#L191 it seems ldt gdt etc effectively don't need to be configured on non BSP

<damo22>pretty sure they do

<damo22>since APs come up on 16 bit mode

<damo22>so you have to reset the selectors

<damo22>and we dont want interrupts to be enabled on APs

<damo22>only traps

<Pellescours>ok

<Pellescours>when I uncomment your "break" code, it hang at configuring gdt step

<damo22>correct, that is the problem i dont know what is wrong

<damo22>maybe it triggers a trap but the traps arent configured in locore.S or something

<Pellescours>is the gdt pointing to the same area accross all cpus?

<damo22>no

<damo22>there are copies made

<damo22>see mp_desc_init()

<damo22>i dont use "gdt" pointer itself to load the gdt on aps

<damo22>i wrote an ap_gdt_init function that uses the per cpu versions

<Pellescours>can it be a problem of ap gdt content pointing to bsp gdt content?

<damo22>maybe but i thought i patched it

<Pellescours>you just memcpy it

<damo22>yes and then below patch it

<damo22>maybe i need to patch everything before calling ap_gdt_init

<Pellescours>right

<Pellescours>maybe

<damo22>you can try moving the fills from individual ap_*_init into mp_desc_init

<damo22>and then call ap_gdt_init

<damo22>or just use memset(0

<damo22>and then fill them one by one

<damo22>that might work

<damo22>instead of memcpy

<Pellescours>damo22: you don't fill the gtd the same way in gdt_fill (gdt.c), may this explain the hang?

<damo22>try

<Pellescours>after more research, i think is that the configuration of lapic is not complete

<damo22>do you think we actually have execution on the APs?

<damo22>i cant seem to set bit 8 of the spurious vector on the APs

<damo22>but they already received an IPI/SIPI

<damo22>that doesnt make sense, how can software set the lapic->spurious vector before the AP starts?

<damo22>so it can receive an IPI/SIPI

<damo22>ie, according to the intel datasheet, its impossible to start an AP if that bit needs to be toggled first

<damo22>on the AP itself

<damo22>On AP 2:

<damo22>ES =0010 40000000 ffffffff 00c09300 DPL=0 DS [-WA]

<damo22>CS =0008 40000000 ffffffff 00c09b00 DPL=0 CS32 [-RA]

<damo22>SS =0010 40000000 ffffffff 00c09300 DPL=0 DS [-WA]

<damo22>DS =0010 40000000 ffffffff 00c09300 DPL=0 DS [-WA]

<damo22>why is the high word of the segment registers 0x4000?

<damo22>on BSP its zero

<damo22>i think i am reading this wrong

<damo22>c10551cc: 0f 01 15 62 51 05 01 lgdtl 0x1055162

<damo22>c10551d3: ea da 51 05 c1 08 00 ljmp $0x8,$0xc10551da

<damo22>c10551da: 66 b8 10 00 mov $0x10,%ax

<damo22>the lgdtl is loading the gdt from the wrong address, its missing the 0xc0000000 offset

<damo22>Pellescours: the gdt is trying to load at an address without the 0xc0000000 offset

<damo22>then it crashes i think

<damo22>youpi: if i define a symbol in a .text section of asm, and i want to refer to it in C, do i need phystokv(symbol) ?

<damo22>for gnumach

<youpi>no

<youpi>asm lives in the virtual world as well

<youpi>the linker doesn't care whether the code comes from C or asm

<damo22>okay!

<civodul>i'm looking at https://issues.guix.gnu.org/58320 and wondering why there's no exec task showing up

<civodul>presumably mach creates it and it terminates prematurely i guess?

<damo22>this cant be right? GDT= 01055140 0000001f

<damo22>doesnt it need to be c1055140?

<damo22>ok its crashing on GDT= c11d8aa4 00000003

<damo22>youpi: can i load a dummy gdt at boot on the AP?

<damo22>i need 2 dummy gdts, one for 16 bit and one for protected mode

<damo22>and then mach loads another one

<youpi>civodul: perhaps set

<youpi>debug_all_traps_with_kdb

<youpi>to catch a potential exec crash in kdb

<youpi>damo22: depends whether it's a linear address or a virtual address

<damo22>in qemu, when i do "info registers"

<youpi>you can probably load some temporary session descriptors during the bootstrap yes

<youpi>I don't know what qemu prints

<youpi>just saying that I wouldn't be surprised that it'd be linear addresses

<damo22>i think its printing linear addresses, because i see 0xf6000 sometimes when its in very early stage

<damo22>and sometimes 01055140

<damo22>so maybe if i see c0000000 addresses its a bug?

<youpi>depends if it's linear or virtual address :)

<damo22>but the BSP has GDT= c11201c0 00000057 when it runs in single mode

<damo22>i think i need to make the APs print similar range of address as BSP in qemu

<damo22>when i allocate the gdt from kernel

<damo22>ahh, when i run qemu with -smp 2 but dont use smp, the second cpu still has a GDT and an IDT

<damo22>loaded from the bios

<damo22>GDT= 000f6180 00000037

<damo22>IDT= 000f61be 00000000

<damo22>these are linear addresses right?

<youpi>most probably yes

<youpi>these are addresses inside the bios area

<youpi>(physical addresses, event)

<youpi>-t

<damo22>why is (-KERNELBASE) used

<damo22>in the boot gdt

<damo22>cant it be 0?

<youpi>I don't remember the details, but possibly the ground reason is that you can't load the kernel at physical address 0

<youpi>ah no that's the base

<youpi>forget what I wrote

<damo22>maybe KERNELBASE should be (-_START) ?

<damo22>c1000010 <boot_entry>:

<damo22>c1000010: 0f 01 15 82 00 00 01 lgdtl 0x1000082

<damo22>c1000017: ea 1e 00 00 c1 08 00 ljmp $0x8,$0xc100001e

<damo22>c100001e: 66 b8 00 00 mov $0x0,%ax

<damo22>so the linear address 0x1000082 has the gdt for bootstrap but it corresponds to kernel address 0xc1000082 ?

*civodul made some progress: https://issues.guix.gnu.org/58320#1

<Pellescours>civodul: dumb question but in guix, are the same patches applied than with the debian version?

<civodul>Pellescours: not necessarily

<civodul>but... there's an "upstream", right? :-)

<civodul>do you have specific patches in mind?

<Pellescours>not in particular

<damo22>i think the gdt is the problem

<damo22>(for smp)

<damo22>the selectors are not the same as the bsp but its tricky to set them correctly without crashing the cpu

<damo22>so the first gdt has 0x0 offset, the second gdt has 0x40 and then when the third gdt is loaded it has an invalid address

<Pellescours>so you success to get gdt for cpu 1 being loaded?

<damo22>Pellescours: the bootstrap process loads a gdt in 16 bit mode, this succeeds, then the protected mode loads another gdt, this works, but the third gdt in mp_desc.c crashes the cpu

<youpi>you are loading it with linear addresses, right?

<Pellescours>yes "pdesc.linear_base = kvtolin(&mp_desc_table[cpu]->gdt);"

<damo22>but the old gdt has special values that offsets the address by 0x40000000

<youpi>the descriptor is using linear addresses, so it doesn't care about offsets in previously-loaded segments

<youpi>and the lgdt instruction itself uses the virtual address, right?

<Pellescours>it does a "lgdt(&pdesc)" pdesc being a struct allocated on the stack (a simple variable)

<youpi>ok

<Pellescours>It’s basically the same step as the same done for BSP

<damo22>yes but i can tell you the offsets in the bsp segments are all zero

<damo22>but when i try to do that on the APs, i cant get past the second lgdt

<Pellescours>I printed the limit and linear address of gdt. I get limit=87 and linear_base=c10b3db4 does this seems correct?

<damo22>hmm i think linear_base should be 10b3db4

<youpi>limit=87 is really not looking correct

<youpi>it'd mean the whole addressing fits in 0x88 bytes :)

<youpi>ah no you mean the pdesc.limit

<damo22>can i kalloc before gdt is set up?

<youpi>0x87 looks a bit bizarre, since descriptors are 64bit long

<youpi>damo22: kalloc is available *way* after that

<damo22>biosmem_bootalloc?

<youpi>init_alloc_aligned, rather

<youpi>since that's what the rest of the code already uses

<damo22>do i need to call phystokv on the return pointer from init_alloc_aligned?

<youpi>I don't know but the source code probably does know

<youpi>I guess it returns virtual addresses

<youpi>but only the source code will know for sure

<damo22>(qemu) cpu 0

<damo22>(qemu) x 0xc11d8aa4

<damo22>c11d8aa4: 0xf51ee800

<damo22>(qemu) cpu 1

<damo22>(qemu) x 0xc11d8aa4

<damo22>c11d8aa4: Cannot access memory

<damo22>that is why it crashes

<damo22>it needs to access the virtual address to compute the linear address

<youpi>? there is no computation of the linear address

<youpi>all that needs to be done is to provide a virtual address according to the segmentation currently in use

<youpi>see boothdr.S's lgdt boot_gdt_descr - KERNELBASE

<youpi>at that point multiboot initialized virtual = linear, so we need to shift

<youpi>but later on lgdt is done on the shifted virtual addresses since that lgdt precisely did set up the shifting (so that almost all gnumach code has virtual addresses shifted)

<youpi>gnu_srs1: rather catenate the gcc patches into just one, since they're completely related (and drop patch 1 indeed, it's useless in the debian package). Also give the patch file a useful name, not a number :)

IRC channel logs

2022-10-06.log