***_zxq9_ is now known as zxq9
<ArneBab_>sneek: later tell nalaginrut: the scoping, the data formats, and the type conversion. <catonano>I just successfully wrapped a C function in Guile scheme and I'm so happy ! ***random-nickname is now known as random-nick
<amz3>I'd like to raise the issue that paroneayea is AFAIK the only one that has access to @guilelang twitter account in particular <amz3>As matter of fact, paroneayea is biased in the regard of what makes a good tweet for guile community <amz3>WDYT should be done to deal with this situation? <paroneayea>amz3: the maintainers also have access but I'm the only one who logs in. but I also don't log in often <amz3>I explicitly highlighted @guilelang it's very unlikely that you missed my tweets related to guile <paroneayea>amz3: I guess you're annoyed that I don't always repost your things, but I have reposted them sometimes? <amz3>same message is toping /r/guile <amz3>Yes I am annoyed that the thing I've been working on 3 years is not reposted yes <amz3>without getting any feedback <paroneayea>hm, I thought I had. I've reposted something that davexunit posted about your blogpost in the past but nothing by you <daviid>hello guilers! I have a serious (performance related) problem using large images, 1GB or more, in Guile-CV, and talking to the vigra and vigra_c folks, they pretend it is a guile problem (typical 'the ball is in your camp':)), which I first very much doubt but then I have to prove this, so I did a few tests, time and profile and need help to interpret some figures... hope super pro profilers will shim in :):), let me expose the <daviid>so, loading a 1GB image tales 5'10" in guile, 3.6 sec in vigrapython and 14" when using the same vigra_c binding but in mathlab (I don't use mathlab of course, I use ocxtave, but the vigra_c developer did that measure ...) <daviid>to complete the figures, loading the exact same image in guile but using a libfreeimage takes 0.9sec <daviid>so, to try to eliminate Guile as the 'culprit, I measured the time taken to allocate the 3 channels of 350MB, f32vectors actually, and this gives us: <daviid>scheme@(guile-user)> ,time (im-make-channels 17711 20208 3) <daviid>$1 = (#f32(0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 …) …) <daviid>;; 0.538621s real time, 1.366837s run time. 1.286892s spent in GC. <daviid>this is on a very powerfull machine by the way, 12 cores 24GB of mem and 2 SSD (so the performance problem is not the hard disk, neither a mem, nor a cpu prob <daviid>,time (im-load Pelota3-20X.tif) gives 5'10" so I profiled <daviid>scheme@(guile-user)> ,profile (im-load "Pelota3-20X.tif") <daviid> 50.00 155.48 155.48 anon #x556937b5b148 <daviid> 50.00 155.48 155.48 make-srfi-4-vector <daviid> 0.00 310.96 0.00 cv/impex.scm:129:0:vigra-load-rgb-image <daviid> 0.00 155.48 0.00 ice-9/threads.scm:284:4:loop <daviid>Total time: 310.960342833 seconds (1.253158522 seconds in GC) <daviid>first, I don't know how to interpret these numbers, I was expecting below a second for allocation and 5'9" spent in the vigra_c cv/impex.scm:129:0:vigra-load-rgb-image <daviid>second, how come 155.48 0.00 ice-9/threads.scm:284:4:loop ? <ijp>because that column is cumulative <ijp>the last column is the important one <daviid>ijp: I did read that but stiull could not 'read' these properly <ijp>so you total at 310 seconds which is 5 m 10 s as expecte <daviid>ijp but 155.48 make-srfi-4-vector does not make sence, it should be 0.5 <ijp>it might be a mistake, it says there were only two samples taken <daviid>and then the rest of the time should be 10 or 11 sec, not 5 min... <daviid>ok, I will just profile vigra-load-rgb-image to see <ijp>where is the code for im-make-channels <ijp>if these allocations are large, could it be a gc issue? <daviid>ijp: no, the allocation time is 0.5 sec <ijp>have you tried precomputing all the arguments, and only timing the call to load-rgb-image <catonano>one of the C functions I have to wrap in Guile scheme expects an "unsigned short" as an argument. How do I indicate that ? uint8 ? <ijp>daviid: have you tried precomputing all the arguments, and only timing the call to load-rgb-image <ijp>catonano: shorts are at least 16 bit, so probably uint16 <ijp>you'd probably need to do configuration to make it portable, idk, which is probably why the types guile supports all specify the size <daviid>ijp: sorry, inadvertently tried to measure im-make-channels ... on my laptop, which literalçly killed it :) <catonano>uhhmm nothing. I can't get it to accept a signature <ijp>daviid: I was just saying, have you tried precomputing all the arguments, and only timing the call to load-rgb-image <daviid>ijp: I was trying to do that, but wrong window, and my laptop does not have the capacity to hold these size, let me try, give me a sec <catonano>ijp: what do you mean, exactly with: "you'd probably need to do configuration to make it portable" ? <ijp>it is possible, but I have no idea how likely, that 'short' can mean different things on different computer systems <ijp>in that case you would want to choose the type depending on the system <ijp>this seems like something (system foreign) should be able to handle <catonano>that maybe the case. uint8, uint16 and uint32 produce the same error: wrong argument type <daviid>scheme@(guile-user)> ,profile ((@@ (cv impex) vigra-importrgbimage-c) r g b width height f) <daviid>100.00 308.88 308.88 anon #x558100d213e8 <daviid>Total time: 308.883311234 seconds (0.0 seconds in GC) <daviid>so, this is, strictly, the time spend executing the vigra_c code ... however they pretend (the vigra_c developer) loading the same image size, using this vigra-importrgbimage-c but from mathlab, takes 14sec, which I doubt, coud that be possible? <ArneBab_>daviid: do you get the same time without ,profile? <ijp>I don't know how, but could there be some (system foreign) overhead? <daviid>ijp: that is what I really would like to know, and happy to learn how to debug this maybe <ArneBab_>then I could imagine that the matlab folks use some shortcuts which avoid restructuring data <ijp>I can believe they might take some shortcuts, but that wouldn't explain why python is so much faster <ArneBab_>Python is pretty good at keeping things down in C <daviid>however, when I loada the same image but using libfreeimage (from davexunit), which also uses (system foreign), I get 0.9sec <ArneBab_>when using numpy it effectively never leaves the C array <ArneBab_>daviid: what kind of datastructure do you get from that call? <daviid>ArneBab_: I allocate, 3 f32vectors of +- 350MB each <ArneBab_>that’s what it returns? What’s the initial size of those vectors? <daviid>ArneBab_: it does notr return anytrhing <ArneBab_>so you pass it the vectors and it populates them? <daviid>ijp: given using libfreeimage takes 0.9sec, using the FFI as well, I take for granted that there is no over time spent in (system foreign), wdyt? <ArneBab_>daviid: which flags are used to compile libvigra? <daviid>ArneBab_: all I do is: cmake -DCMAKE_INSTALL_PREFIX=/opt/vigra_c; make; make install <daviid>ArneBab_: right, and Benjamin (the author) said he profiles and it takes 14" <ArneBab_>can you see the CFLAGS when you execute ccmake . in vigra_c? <ArneBab_>ccmake should give you an interactive UI <ArneBab_>it doesn’t come with cmake? (the world changed…) <daviid>ArneBab_: don't know but command not found ... <ArneBab_>I’d first need to install vigra to make that work … <daviid>ArneBab_: cmake-gui does not list CFLAGS <daviid>is there a cmake option to just lkist the flags? <ArneBab_>with that you can check the actual compiler calls — and see whether it uses optimization <daviid>here is the first compilation 'report': <daviid>cd /usr/local/src/vigra_c/git/src && /usr/bin/c++ -Dvigra_c_EXPORTS -I/opt2/vigra/include -fPIC -std=gnu++11 -o CMakeFiles/vigra_c.dir/vigra_convert_c.cxx.o -c /usr/local/src/vigra_c/git/src/vigra_convert_c.cxx <ArneBab_>that looks like it does not use any optimization at all <ArneBab_>I’d have expected to see at least -O2 in there <daviid>ArneBab_: trying your D... suggestion, let'1s see <ArneBab_>(though it should still be fast if vigra is compiled with optimization turned on) <daviid>I don't know if vigra is compiled with optimization because I manually compiled it <daviid>/usr/bin/c++ -Dvigra_c_EXPORTS -I/opt2/vigra/include -O3 -DNDEBUG -fPIC -std=gnu++11 -o CMakeFiles/vigra_c.dir/vigra_convert_c.cxx.o -c /usr/local/src/vigra_c/git/src/vigra_convert_c.cxx ... <daviid>maybe I should recompile vigra first <ArneBab_>that can make several orders of magnitude difference <daviid>ArneBab_: thanks, I think I should recompile vigra <daviid>let see it will only take a few minutes <daviid>scheme@(guile-user)> ,profile ((@@ (cv impex) vigra-importrgbimage-c) r g b width height f) <daviid>100.00 4.07 4.07 anon #x55ef8f4a1c28 <ArneBab_>daviid: you might want to use type RELWITHDEBINFO while developing <daviid>ArneBab_: thanks but I don't develop vigra neither vigra_c, I bind these... though thanks for the tip <ArneBab_>these are just to get good backtraces when something goes wrong