Log file Viewer

[15:03:44] <techair> hello. i am calling tobytes() on an image to pass off to a C library that performs image operations. I think tobytes() strips the header information of the image because the library fails to parse it. If I save the image to a StringIO object, then call .getvalue() on that object, it works. But, I am trying to pass the image from memory instead of saving it

[15:03:44] <techair> to disk or saving it to StringIO. Does pil offer a way for me to pass only the raw data (including header) into a variable?

[15:36:58] <wiredfool> techair: tobytes is raw image bytes

[15:37:15] <wiredfool> so it would be uncompressed rgb(a) bytes

[15:37:26] <techair> wiredfool: just read that in the docs.. trying to pass an encoder into it for "jpeg", and its complaining about that

[17:36:12] <techair> wiredfool: i fixed that portion

[17:37:47] <techair> i made it const char *mem and it compiles

[17:38:29] <techair> err, &mem in that argument list

[17:42:19] <wiredfool> so, that might work in py2.x.

[17:42:47] <wiredfool> you'd have to be passing in something that's essentially a python string

[17:43:21] <wiredfool> you might be able to pass in a stringio/buffer interface object and get it's address

[17:47:46] <techair> yea wiredfool it works with StringIO

[17:48:09] <techair> the issue is i am trying to optimize it so I can load an image from python and send it to this C program without using StringIO, having I/O constraints

[17:48:45] <techair> so i pull an image from the web, and insted of saving to StringIO or to Disk to pass this into the library, i want to call get_cvec_from_mem with just the raw data loaded from pillow

[18:02:19] <wiredfool> there are 3 ways an image can be represented:

[18:02:34] <wiredfool> 1) encoded, likely jpeg, png, or gif if you're grabbing from the web

[18:03:00] <wiredfool> 2) raw bytes -- or what you get with tobytes after loading and decoding the image in stage 1

[18:03:38] <techair> hmm.. ok. So maybe I dont decode it?

[18:03:43] <wiredfool> 3) the internal pillow format, which is really one of a couple memory layouts and behind an api wall

[18:04:26] <wiredfool> are you running on linux?

[18:04:31] <techair> yes

[18:04:37] <techair> Service runs it on ubuntu, im on OSX

[18:05:00] <wiredfool> if you want to blow away most io, you might be able to use a tempfs for a filesystem

[18:05:13] <wiredfool> might be in /dev/shm

[18:05:20] <wiredfool> might be something you have to mount

[18:05:24] <wiredfool> but it's like a ramdisk

[18:05:43] <wiredfool> likely to cut out any io bottlenecks without too much pain

[18:05:51] <techair> ah! ok

[18:05:56] <techair> This might be better..

[18:06:00] <techair> Instead of C hacking :)

[18:06:22] <techair> it'll be only for storing temporary image files, they are wiped and turned into vectors after it completes

[18:06:25] <techair> so i think the use case is OK

[18:06:44] <wiredfool> it's backed by swap. You don't want to use more than half your ram

[18:07:58] <wiredfool> if that doesn't get you far enough, then you could dig deeper into the pipeline. But my guess is that you'd be hitting processor or download limits first

[18:10:28] <techair> yea