pmxbot IRC Log Viewer

[17:22:27] <AleXoundOS_> Hi. I'm converting .pcf font to .pil font. When I try to access any character which number is greater than 128 I get wrong characters.

[17:27:55] <wiredfool> That's interesting. I haven't don't much with the .pil fonts

[17:28:17] <wiredfool> what version are you running

[17:28:45] <AleXoundOS_> let me check

[17:29:52] <AleXoundOS_> I'm not really sure... I use Python 3 and thus it's not pure PIL but PILLOW

[17:29:57] <wiredfool> ok.

[17:30:00] <AleXoundOS_> I'm not sure if you support PILLOW

[17:30:07] <wiredfool> yep, that's what I work on

[17:30:14] <AleXoundOS_> nice

[17:30:16] <AleXoundOS_> version is 2.2.1-2

[17:30:32] <AleXoundOS_> without "-2"

[17:31:16] <AleXoundOS_> so when I print characters with chr(i)

[17:31:27] <AleXoundOS_> and i is in range from 1 to 128

[17:31:28] <AleXoundOS_> it's fine

[17:31:33] <AleXoundOS_> I get correct characters

[17:31:49] <AleXoundOS_> but everything higher 129 doesn't have a clear logic

[17:31:55] <wiredfool> ah

[17:31:55] <AleXoundOS_> higher 128*

[17:32:19] <wiredfool> It looks like freetype supports .pcf fonts

[17:32:35] <AleXoundOS_> how can I use it?

[17:32:37] <wiredfool> have you tried just pointing imagefont at the .pcf?

[17:32:48] <AleXoundOS_> Image.Font.?

[17:32:52] <AleXoundOS_> ImageFont.?

[17:32:58] <wiredfool> sec

[17:33:26] <wiredfool> https://github.com/python-imaging/Pillow/blob/master/Tests/test_imagefont.py#L48

[17:33:36] <wiredfool> tho, that's the truetype

[17:33:49] <wiredfool> let me see where to pull in pcf

[17:34:47] <AleXoundOS_> (I got .pcf after gunzip'ing /usr/share/font/misc/*.pcf.gz font)

[17:35:39] <wiredfool> you know, I'd just try throwing it through ImageFont.truetype. I don't think that it's actually checking for a font type, just dumping it off on freetypw

[17:37:07] <AleXoundOS_> wiredfool, http://pastebin.ca/2579079

[17:38:59] <wiredfool> bah. ok, it's more complicated than that

[17:39:41] <AleXoundOS_> wiredfool, ah, wait, sorry I tried .pil font

[17:40:24] <AleXoundOS_> wiredfool, with .pcf I get this: http://pastebin.ca/2579112

[17:41:29] <wiredfool> ok, at least it's _different_

[17:42:20] <AleXoundOS_> any ideas?

[17:42:28] <wiredfool> looking @ code

[17:42:36] <AleXoundOS_> ok:)

[17:43:05] <wiredfool> try passing in 7 or 13 for the size

[17:43:20] <AleXoundOS_> ok

[17:43:25] <wiredfool> so ImageFont.truetype(fn, size=7)

[17:44:08] <AleXoundOS_> 13 worked, but I get empty squares in the output

[17:44:56] <AleXoundOS_> I tried range from 1 to ~100

[17:45:10] <AleXoundOS_> all of them are empty squares

[17:47:44] <wiredfool> ok.

[17:48:27] <AleXoundOS_> btw, the empty square it draws is really from .pcf font

[17:48:36] <AleXoundOS_> it's the first character

[17:48:47] <wiredfool> interesting

[17:49:06] <AleXoundOS_> maybe I need to use some special encoding?

[17:50:15] <wiredfool> it's possible you're getting bit by unicode

[17:51:24] <AleXoundOS_> I tried chr(i)

[17:51:33] <AleXoundOS_> where i was in range from 1 to 100

[17:51:50] <AleXoundOS_> and tried 'A' too

[17:53:24] <wiredfool> hmm.

[17:54:06] <wiredfool> you just used pilfont to convert the original font?

[17:54:20] <wiredfool> thinking about the original problem now, since freetype wasn't an easy solution

[17:55:24] <AleXoundOS_> yes I used pilfont to convert .pcf font into .pil font (it generated .pbm as well)

[17:55:54] <wiredfool> right. The .pil is the metrics file, the .pbm is a bitmap of the glyphs

[17:56:01] <AleXoundOS_> yea

[17:56:20] <AleXoundOS_> .pbm file looks to be correct

[17:56:43] <AleXoundOS_> it contains the characters I need after 128

[17:57:09] <wiredfool> ok, showing how much I've looked at this section of the code, have you tried reading the .pcf using PcfFontFile?

[17:57:25] <AleXoundOS_> no, I didn't know it exists, I will try

[17:57:26] <wiredfool> https://github.com/python-imaging/Pillow/blob/master/Tests/test_font_pcf.py

[17:57:33] <wiredfool> yeah, neither did I

[17:57:45] <wiredfool> there's even a test for it.

[18:06:54] <AleXoundOS_> eh, I couldn't find "tester" module

[18:07:13] <AleXoundOS_> I suppose it's used for "tempfile" function

[18:12:50] <AleXoundOS_> wiredfool, well, I tested

[18:13:14] <AleXoundOS_> wiredfool, first it produces exactly the same .pil and .pbm files

[18:13:19] <wiredfool> tester is in tests/

[18:14:05] <AleXoundOS_> it converts pcf to same pair or .pil and .pbm files and then loads .pil

[18:14:09] <AleXoundOS_> result is the same

[18:14:38] <AleXoundOS_> I get wrong characters after trying to access anything higher than 128th char

[18:15:21] <wiredfool> ok, then I'm guessing it's picking out the metrics incorrectly

[18:15:55] <AleXoundOS_> for example instead of ё character, I get "T" character for the next 64

[18:16:20] <AleXoundOS_> so range 128-192 are all "T"

[18:17:03] <AleXoundOS_> and occasionally it prints some random second character after "T"

[18:22:15] <AleXoundOS_> wiredfool, here is the .pbm file: http://i.imgur.com/MX1HFma.png

[18:22:38] <AleXoundOS_> wiredfool, and this is what I get iterating chr(i) from 1 to 224 http://i.imgur.com/vs1UbVP.png

[18:30:55] <wiredfool> ok. I'm wondering if your font is just messed up. I've grabbed a copy of that font from ubuntu 12.04lts, (7x13-ISO8859-5.pcf), and I'm getting something reasonable

[18:31:17] <AleXoundOS_> hm

[18:31:39] <AleXoundOS_> can you link me your pcf file?

[18:32:47] <wiredfool> https://gist.github.com/wiredfool/8584125 is the code

[18:35:12] <AleXoundOS_> well, I tried:

[18:35:13] <AleXoundOS_> font = PcfFontFile.PcfFontFile(file)

[18:35:17] <AleXoundOS_> font.save(tempname)

[18:35:48] <wiredfool> https://www.dropbox.com/sh/hvblk3cee39fbmx/wTjWwxIqfC

[18:36:09] <wiredfool> I think that's the public link

[18:36:42] <AleXoundOS_> thx, I will try your pcf file

[18:41:08] <wiredfool> can you post your pcf and .pil files?

[18:42:51] <AleXoundOS_> yeah, but I downloaded your pcf and it's exactly the same

[18:43:32] <AleXoundOS_> pil file: http://rghost.net/51875640

[18:46:01] <wiredfool> hmmm

[18:48:02] <AleXoundOS_> how can I easily import "tests" from downloaded git folder?

[18:49:53] <wiredfool> so, I run it from the Pillow directory, python Tests/test_foo.py --installed

[18:50:05] <wiredfool> it's _really_ touchy

[18:51:04] <AleXoundOS_> ok, I managed to run your test

[18:51:25] <AleXoundOS_> first of all I get:

[18:51:25] <AleXoundOS_> test.py:22: assert_equal(len([_f for _f in font.glyph if _f]), 192) failed:

[18:51:31] <AleXoundOS_> - got 223, expected 192

[18:51:50] <wiredfool> right. I didn't touch that up, I was just using it for the conversion. sorry

[18:52:07] <wiredfool> now, I tried it on 3.2, and I'm getting what you got.

[18:52:30] <wiredfool> or at least, I'm getting it wiht lots of T

[18:52:32] <wiredfool> and Y

[18:52:42] <wiredfool> and other funkiness.

[18:52:48] <AleXoundOS_> yes

[18:53:29] <AleXoundOS_> wiredfool, this is what I get with your test: http://rghost.net/51875857

[18:53:41] <wiredfool> ok, not getting exactly what you're showing, but close enough. It's a py3 regression.

[18:53:46] <wiredfool> and possibly unicode

[18:54:39] <wiredfool> I've got enough here to make a reproducable test case

[18:55:34] <AleXoundOS_> heh, what options do I have to make it work? downgrading to Python 3.2 is the only option you think?

[18:57:17] <wiredfool> It'll work on 2.7 now. I don't think it's going to be that hard to patch

[18:57:23] <wiredfool> there may be some workaround

[18:57:45] <AleXoundOS_> oh, 2.7 is not an option at all))

[19:01:23] <wiredfool> wild guesses here -- I'm thinking that it's probably a unicode encoding issue, and that the problem isn't in the font creation but in the actual drawint.

[19:04:12] <wiredfool> I'm also out of time to work on it this morning.

[19:04:25] <aclark> wiredfool: man the Pyramid shop has been busy

[19:04:40] <wiredfool> I'll probably be a few days to track it down.

[19:04:59] <wiredfool> aclark: ?

[19:05:49] <aclark> wiredfool: getting lots of github emails lately

[19:07:50] <AleXoundOS_> wiredfool, anyways, thank you very much, I've ideas on how to access the rest part at least, your test showed that there are usable characters somewhere between ~160-192

[19:09:30] <wiredfool> AleXoundOS_: I'm sure it's fixable It just needs going through each path and seeing where they diverge

[19:10:51] <wiredfool> aclark: so, that pypy segfault -- turns out it is a real use after free issue in our code

[19:13:30] <AleXoundOS_> wiredfool, there is some pattern in the "chaos" of characters, I tried with range(2230) http://rghost.net/51876417

[19:15:32] <wiredfool> what locale are you in?

[19:15:37] <wiredfool> or using?

[19:15:55] <wiredfool> wait, I'm getting it too, so that can't be it

[19:17:59] <wiredfool> There shouldn't be any characters in the font above 255, so somewhere there's a conversion that's taking a byte stream, interpreting it as unicode, and doing stupid things with it.

[19:18:26] <AleXoundOS_> yea, sounds logical

[19:21:51] <AleXoundOS_> unfortunately the pattern breaks somewhere after "п" character, I cannot follow it after ~21000 pixel of the generated image

[19:49:02] <aclark> wiredfool: ah

Log file Viewer

Help | Karma | Search:

#pil logs for Thursday the 23rd of January, 2014