PMXBOT Log file Viewer

Help | Karma | Search:

#mongodb logs for Sunday the 18th of May, 2014

(Back to #mongodb overview) (Back to channel listing) (Animate logs)
[00:45:11] <wladston> Mongodb isn't validating this Polygon when I try to create the geospatial index: http://pastebin.com/j2SMb4wE but http://geojsonlint.com/ validades the polygon. Any ideas of what might be going wrong?
[01:17:24] <ehershey> wladston: that's really weird
[01:18:56] <wladston> ehershey: I've been trying to fix this problem since yesterday
[01:18:59] <wladston> full time
[01:19:32] <wladston> I'm trying to import data from a shapefile… I've wiped everything and tried again
[01:19:57] <wladston> not it complains about another polygon, but geojsonlint.com still shows it as valid
[01:20:04] <wladston> *now
[01:20:28] <ehershey> I don't think I can figure out exactly what the problem is
[01:20:37] <wladston> I think it might be related with the polygon not starting and ending with the same coordinates
[01:20:43] <ehershey> ohh
[01:21:02] <wladston> I'm trying to research on how I can automatically fix this problem for all polygons
[01:22:25] <wladston> ehershey: do you know anyone who would help me in this? I'm kinda desperate :/
[01:22:27] <ehershey> I don't think that's the problem
[01:22:38] <wladston> I don't think that's the problem too
[01:22:43] <ehershey> I can get an error in a mongo shell just trying to create a variable containing your coordinates array
[01:22:44] <wladston> but I don't know what else to try
[01:22:47] <ehershey> and it should be a valid javascript array
[01:23:16] <ehershey> I think you can get more people to look at the problem if you email the mongodb-user mailing list
[01:23:17] <wladston> ehershey: let me paste the lastest coordinate array, hold on
[01:24:40] <wladston> ehershey: here, please, take a look
[01:24:42] <wladston> http://pastebin.com/b5QgS1GW
[01:24:52] <wladston> http://jsonlint.com/ will say that this is a valid json
[01:25:14] <wladston> http://geojsonlint.com/ will say that this is a valid polygon (and will even draw it on the map)
[01:25:52] <wladston> and I took this polygon right from the mongodb error message that says "Can't extract geo keys from object, malformed geometry?"
[01:26:34] <ehershey> I recommend reducing your error case to the smallest possible snippet that gives you unexpected behavior
[01:26:35] <wladston> I've checked, all the coordinate points are within the valid boundaries, the longitude/latitude order is correct…
[01:27:10] <ehershey> and posting here: https://groups.google.com/forum/#!forum/mongodb-user
[01:27:19] <wladston> ehershey: how could I do that? manually removing points from the polygon?
[01:29:07] <ehershey> well for example, this gives an error:
[01:29:33] <wladston> ehershey: witch error? where?
[01:29:44] <ehershey> http://pastebin.com/q656qb4D
[01:29:45] <ehershey> sorry
[01:29:49] <ehershey> it took me a sec
[01:30:00] <ehershey> that's a subset from your initial pastepin
[01:30:05] <ehershey> without any geo stuff
[01:30:08] <ehershey> and it should be a valid array
[01:30:29] <wladston> ehershey: humm, I see
[01:30:54] <wladston> ehershey: whoa, any ideas on how can I spot where is the problem in that array?
[01:32:33] <ehershey> no I dunno
[01:33:34] <wladston> when I paste the array in a normal javascript shell it evaluates without error
[01:36:26] <wladston> ehershey: I this problem might be related to a long array being parsed. I'll generate a long array with only the number 1, let's see if the shell tells the same error
[01:36:56] <ehershey> I tried that
[01:37:01] <ehershey> just replaced every number with 1
[01:37:03] <ehershey> and the shell took it
[01:37:05] <wladston> and it worked?
[01:37:10] <wladston> Oo
[01:37:10] <wladston> wow
[01:37:25] <ehershey> I thikn it's a bug
[01:37:30] <ehershey> but I'm not sure
[01:39:48] <wladston> ehershey: I replaced all the negatives (-) for nothing, and it worked
[01:39:58] <ehershey> so weird
[01:41:06] <wladston> ehershey: actually if we remove the negative sign from only the 8 first values it will work
[01:41:32] <ehershey> if I took out the first coordinate pair array it worked too
[01:41:51] <ehershey> I'm going to open a jira ticket
[01:42:32] <wladston> yes
[01:42:35] <wladston> that's really strange
[01:42:53] <wladston> ehershey: you're the man, thanks :)
[01:43:14] <wladston> also please send me the link so I can upvote it
[01:44:37] <ehershey> sure thing
[01:47:40] <wladston> ehershey: so I started to delete these polygons with errors, and now mongo gave me another error, but this time the polygon list evaluates correctly in the shell
[01:47:55] <wladston> it means that the problem probably isn't related to this bug
[01:48:29] <wladston> here is a parte with the error: http://pastebin.com/5XV1bD6a
[01:50:09] <ehershey> I wonder if it realy is the same error
[01:50:14] <ehershey> er, the same problem
[01:50:42] <wladston> ehershey: here is a paste just from the polygon
[01:50:43] <wladston> http://pastebin.com/pmmD1CYE
[01:53:48] <ehershey> for that, I would mail the mailing list
[01:54:11] <ehershey> because it might be something simpler
[01:54:18] <ehershey> like you mentioned, the start/end matching or not matching
[01:55:08] <ehershey> but I don't know
[01:55:17] <wladston> ehershey: I have to deliver this thing in 3 days
[01:55:27] <wladston> ehershey: I'm screwed :(
[01:58:52] <ehershey> I wonder if the polygon is intersecting itself
[01:58:58] <ehershey> just looking at this - http://docs.mongodb.org/manual/core/2dsphere/#geojson-polygon
[01:59:04] <ehershey> For Polygons with a single ring:
[01:59:04] <ehershey> The ring cannot self-intersect.
[02:00:38] <ehershey> here's the jira I made but it may not cover your index error - https://jira.mongodb.org/browse/SERVER-13980
[02:02:34] <wladston> ehershey: well, looking at the picture from the polygon made by geojsonlint, it looks like the polygon doesn't self-intersect
[02:03:04] <ehershey> oh that's good
[02:03:09] <ehershey> I'll play with it a little more
[02:03:16] <ehershey> I reproduced your error on my machine
[02:03:17] <wladston> upvoted
[02:04:44] <wladston> I tried a different strategy. Created another collection, added the index to it, and now I'm copying everything from the original collection to the new one with already has the index
[02:04:57] <wladston> I think at some point the shell is probably going to return an error
[02:05:03] <ehershey> if that works then I think you definitely have a bug
[02:05:15] <wladston> in with case I'll write a small python script to handle these
[02:05:29] <wladston> because this collection has like 30 thousand polygons
[02:05:49] <wladston> and a few ones missing won't cause much problem
[02:06:55] <ehershey> cool
[02:06:58] <wladston> Already over 13% of the polygons were copied without issue so far
[02:07:28] <wladston> so either mongo is being silent about the errors, or we have a bug
[02:07:44] <ehershey> there's a weird nubbin on that polygon on geojsonlint.com
[02:07:48] <wladston> this is what I issued: db.brasil.find().forEach(function(p) { db.brasil3.insert(p); });
[02:07:56] <ehershey> in the northeast corner
[02:08:06] <wladston> with one, the last one I pasted?
[02:08:30] <wladston> OH, yes, I see it
[02:09:33] <ehershey> maybe it does intersect itself
[02:09:39] <ehershey> but the site should validate that I'd think
[02:10:01] <ehershey> I'm sorry I have to go but if you have more problems you should mail the mailing list, a lot of people monitor it
[02:10:09] <ehershey> people who know more about geo stuff than I do :)
[02:10:26] <wladston> right man :)
[02:10:32] <wladston> thanks a lot for the help
[02:10:33] <wladston> really
[02:10:39] <wladston> when you talk to someone it also helps :)
[02:10:48] <wladston> will send that email
[02:13:19] <wladston> vou pesquisar pra ver se tem algo no qgis que conserta isso
[02:24:24] <ehershey> no prob!
[02:24:35] <ehershey> sorry I couldn't help more
[02:28:21] <wladston> ehershey: you helped a lot. I finally made it :) maybe if you weren't here to encourage me I would just have gone to the bar lol :D
[07:17:01] <Favorlock> Hey cheeser, do you have any example or recommendations of how to effectively ensure that all values of a pojo are initalized with morphia or some other library? Essentially I would like to have the ability to add new data to documents with having minimal boiler plate code.
[12:08:33] <salty-horse> hey. using pymongo's update() with multi=true, is there a way to tell it to continue the update process if it encountered some duplicate key errors? insert() has a continue_on_error, but I can't find it in update()
[22:02:31] <kongthap> hi, it's my first time using mongod --config http://pastie.org/9187841, i got parse error on line, please help what went wrong
[22:21:45] <ap4y> yeah, only spaces
[22:22:03] <ap4y> dbpath in you config should be in snake case (dbPath)
[22:22:10] <kongthap> ap4y, what is the appropriated numbers of spacing for indentation ?
[22:22:48] <kongthap> ap4y, i will re-check all typo again, i got line2 error, i just cannot go any more further
[22:22:49] <ap4y> I guess there is no restrictions in yaml format, I personally prefer 2 spaces
[22:23:38] <ap4y> just fix tabs and dbPath and it will work
[22:23:45] <ap4y> at least it works for me :)
[22:25:08] <kongthap> ap4y, thanks i will try with 2 spaces, it's my first time using YAML
[22:27:57] <kongthap> ap4y, more error on line 3 column (it's m) of "c:\mongodb\..."
[22:28:20] <kongthap> ap4y, column 14
[22:28:41] <kongthap> for the file path do i need \\ ?
[22:29:34] <ap4y> Yeah I changed path to be relative (since I'm on MacOS), not sure how it's supposed to be on Windows
[22:30:00] <ap4y> Unix systems use '/' (forward slash) as path separators
[22:30:01] <kongthap> ap4y, i cannot belive that i need \\ instead of \ on windows
[22:30:35] <ap4y> I guess you could try to use /mongodb/some/path instead of c:/mongodb/some/path
[22:31:05] <ap4y> it will probably resolve to your current volume
[22:31:17] <digiv> I am having issues with mapreduce. http://pastebin.com/LVWQsJsr It seems I can't access the variables in doc. The only item that has values from reduce is count.
[22:33:33] <kongthap> ap4y, it's work with c:\\xxx\\yyy
[22:33:50] <ap4y> oh, ok, good to know :)
[22:34:02] <kongthap> i got ** WARNING --jsonp is specified without --httpinterface ** but i did net.http.enabled -> true
[22:37:57] <ap4y> kongthap: not sure about this one, I'm getting it too. It's quite strange because according to the docs: JSONPEnabled option enables the HTTP interface
[22:38:47] <ap4y> it's just a warning, so you shouldn't worry too much, I've never used web interface before tbh :)
[22:39:19] <kongthap> ap4y, i tried to disable both http, jsonp, rest nothing error but the mongod does not start :(
[22:40:22] <kongthap> if the config file is corrent, after mongod --config xxx.conf must start the mongod right?
[22:40:41] <ap4y> what happens? is just exits without error?
[22:41:58] <ap4y> if it's just exits than, there is a chance that it's working(check active processes) or it failed(check log)
[22:42:02] <kongthap> ap4y, i have no ideas :(
[22:42:17] <ap4y> then*
[22:43:35] <kongthap> it's because keyFile: "c:\\mongodb\\data\\key.pem", if deleted this line it just starts
[22:43:58] <kongthap> what is keyFile, do i need to create a file before refer to it ?
[22:44:37] <ap4y> you can find all configuration parameters with descriptions here: http://docs.mongodb.org/manual/reference/configuration-options/
[22:44:39] <kongthap> right now i just have path, no file with the name key.pem
[22:46:25] <ap4y> you probably don't need keyFile confutation, it's related to ssl encryption for connections
[22:47:39] <kongthap> ap4y, all i need now is to set authentication, i just follow tutorail, never known before i don't need the key file, thanks :)
[22:49:35] <ap4y> no problem :)