pmxbot IRC Log Viewer

[02:49:59] <akiba__> hey guys can you help me rewrite sql queries?

[05:15:41] <jarrett> What is mongo's process name?

[05:16:08] <jarrett> I can't seem to get it up an running although I don't recieve any errors from mongod

[05:20:07] <jarrett> What is mongo's process name?

[05:20:33] <LouisT> mongod?

[05:20:44] <jarrett> That's what thought.

[05:20:51] <jarrett> I can't find it after starting up

[05:24:29] <jarrett> where is data/db supposed to go? ~/?

[05:37:26] <mib_52tk8f> Hi.

[05:37:44] <mib_52tk8f> Can we ask mongo support here?

[05:55:21] <akiba__> hey guys

[05:58:39] <stefandxm> hello

[06:00:38] <akiba__> whats up

[06:00:59] <stefandxm> trying to figure out the rfc for SCRAM

[06:01:45] <akiba__> is it 5082?

[06:02:03] <stefandxm> yeah

[06:02:06] <stefandxm> no

[06:02:08] <stefandxm> 5802

[06:02:18] <akiba__> thats what i meant

[06:02:19] <akiba__> lol

[06:02:21] <stefandxm> =)

[06:02:42] <stefandxm> trying to fix my sleeping rythm since the us trip

[06:02:43] <akiba__> do you know java at all?

[06:02:51] <stefandxm> trying not to

[06:02:54] <akiba__> and i live in the us :P

[06:03:07] <stefandxm> where?

[06:03:09] <akiba__> new york

[06:03:16] <stefandxm> oh cool, were you at mongodb world?

[06:03:23] <akiba__> nope

[06:03:32] <akiba__> dont go to the conventions

[06:03:40] <akiba__> and dont feel like being the only teenager there lol

[06:03:42] <stefandxm> it was ok. a bit too crowded tho

[06:03:45] <stefandxm> well

[06:04:03] <stefandxm> for once i think the old guys with suits were the one who felt .. well not fitting in ;)

[06:08:48] <stefandxm> wow http://securityinthe.net/2013/07/26/understanding-sasl-scram/

[06:08:49] <akiba__> lol

[06:08:54] <stefandxm> i love when you find links like that

[06:08:56] <akiba__> guess how old i am

[06:08:59] <stefandxm> esp after having not been sleeping

[06:09:21] <stefandxm> its kindof hard over the internet ;)

[06:11:53] <akiba__> hey stefandxm

[06:11:57] <akiba__> do you know sql at all?

[06:13:31] <stefandxm> could say that

[06:14:37] <akiba__> take a look at this

[06:14:42] <akiba__> and ik it has some java in it lol

[06:14:43] <akiba__> http://pastie.org/9336793#

[06:14:52] <akiba__> but look at the bottom where the queries are

[06:16:47] <stefandxm> ouch

[06:16:54] <stefandxm> dont do that

[06:17:02] <stefandxm> use parameters

[06:17:07] <akiba__> what do u mean

[06:17:12] <stefandxm> instead of concating values

[06:17:18] <stefandxm> "DELETE FROM `skills` WHERE playerName = '"+c.playerName+"';"

[06:17:19] <akiba__> confused?

[06:17:32] <akiba__> what should it be?

[06:17:47] <stefandxm> w8

[06:17:53] <akiba__> aight

[06:18:22] <stefandxm> ok

[06:18:23] <stefandxm> its ?

[06:18:25] <stefandxm> in jdbc

[06:18:26] <stefandxm> so..

[06:18:37] <stefandxm> "DELETE FROM `skills` WHERE playerName = ?;"

[06:18:43] <stefandxm> and then you should add a parameter

[06:18:48] <akiba__> why the ?

[06:18:49] <stefandxm> with c.PlayerName

[06:18:52] <stefandxm> its a parameter

[06:19:05] <stefandxm> if you do + <variable> you will be error prone to attacks such as sql injections

[06:19:05] <akiba__> so how do i add the parameter to that?

[06:19:58] <stefandxm> PreparedStatement stmt = con.cprepareStatement("select * from SRC_AD_USERS where USER_NAME ='?' ");

[06:19:58] <stefandxm> stmt.setString(1,request.getParameter());

[06:19:58] <stefandxm> ResultSet rs = stmt.executeQuery();

[06:20:05] <stefandxm> (stolen from random post on the internet)

[06:20:42] <stefandxm> you should do this for all your + <variables>

[06:21:07] <akiba__> then how do i have the game set the ? parameter

[06:21:14] <akiba__> so it sends the database the info?

[06:21:17] <stefandxm> no

[06:21:20] <stefandxm> yes

[06:21:25] <stefandxm> check the stmt.setString

[06:21:36] <stefandxm> there is operations for set Integer / Doubles/Floats aswell

[06:21:36] <sellout> akiba__: the setString sets the parameter value.

[06:21:39] <stefandxm> so you are type safe

[06:21:42] <akiba__> 1,request.getParameter()

[06:21:51] <stefandxm> you should check a tutorial for this

[06:22:07] <stefandxm> so you get this right from the start. its very dangerous/bad to guess in this type of situations

[06:22:16] <sellout> akiba__: But, you definitely want to be doing prepared statements rather than concatenation.

[06:22:24] <akiba__> i need a sql coder lol

[06:22:25] <sellout> Yeah, safer and faster.

[06:22:43] <akiba__> anyone wanna help?

[06:23:21] <stefandxm> http://www.vogella.com/tutorials/MySQLJava/article.html

[06:23:39] <sellout> With prepared statements, the DB also knows that you’re making the same request with a changed parameter, so the execution plan stays cached and gets rerun. If you just send a new string, the DB can’t tell that it’s the same request with a different parameter.

[06:23:42] <stefandxm> he shows you how to do prety much everything with that code

[06:24:02] <stefandxm> sellout, i wish that was true :)

[06:24:16] <stefandxm> sellout, ive seen many databases handling pure sql better than prepared statements :(

[06:25:06] <sellout> stefandxm: Really?

[06:25:17] <stefandxm> wrote an ETL software once that pushed quite a lot of data through pure sql and had to rewrite from prepared statements to stringbuilder crap

[06:25:32] <stefandxm> quite horrible. but the speed gain was insane in a couple of databases/drivers

[06:25:52] <sellout> What DBs?

[06:25:55] <stefandxm> but its really a last resort

[06:26:01] <stefandxm> sql server and postgre

[06:26:09] <stefandxm> oracle behaved well

[06:26:12] <akiba__> and i dont think pure sql would work in this case

[06:26:22] <sellout> Hrmm, Postgres shouldn’t behave like that … MS, well …

[06:26:25] <akiba__> cause i need it to be

[06:26:34] <stefandxm> postgres odbc drivers arent not so cool

[06:26:35] <akiba__> when player x logs out his info gets updated

[06:26:43] <stefandxm> postgre want you to use their own libraries

[06:26:52] <stefandxm> MS runs a nasty stored procedure

[06:27:19] <stefandxm> postgre performed beautifully in jdbc, with odbc not so

[06:27:19] <akiba__> but yeah guys so

[06:27:33] <akiba__> i think i need prepared statements

[06:27:42] <stefandxm> yeap

[06:27:44] <stefandxm> thats step #1

[06:27:47] <akiba__> cause i have to automatically update it on user logout

[06:27:55] <stefandxm> also, you should not delete / insert. you should use upserts

[06:28:12] <akiba__> upserts?

[06:28:15] <akiba__> what are they

[06:28:18] <stefandxm> its more of a pattern/idea

[06:28:21] <stefandxm> its update and insert

[06:28:36] <stefandxm> so if a user exist, update him

[06:28:37] <akiba__> so how do i change the code to do that

[06:28:41] <stefandxm> if he doesnt exist, add him

[06:28:56] <akiba__> ik that makes logical sense

[06:29:01] <akiba__> but i have no clue how to do that

[06:29:08] <stefandxm> you could do it in many ways. either in raw sql but it gets nasty

[06:29:26] <stefandxm> so if you are new to sql rather play it safe, check if he exist by a select first. if he doesnt. insert, if he does; update

[06:30:32] <akiba__> how do you do it in raw sql?

[06:31:36] <akiba__> no clue what it means

[06:31:39] <akiba__> but im guessing like this?

[06:31:40] <akiba__> WITH sel AS ( SELECT id FROM categories WHERE name=$1 AND source=$2 ), ins AS ( INSERT INTO categories (name, source, created_at, updated_at ) SELECT $1, $2, 'now', 'now' WHERE NOT EXISTS (SELECT * FROM sel) RETURNING id ) SELECT id FROM ins UNION ALL SELECT id FROM sel;

[06:32:43] <stefandxm> it depends one dialect

[06:33:08] <stefandxm> you could do it like that or with a variable/if

[06:33:29] <akiba__> dont know how to do that lol

[06:33:36] <akiba__> so i guess update and insert

[06:33:43] <akiba__> which i still dont know how to do lol

[06:33:44] <stefandxm> like.. select into a count(1) from users where name = 'blaha'; if a > 0 update (...) else insert(..)

[06:33:49] <stefandxm> but as i said

[06:33:51] <stefandxm> dont start there

[06:33:55] <stefandxm> start by doing it from your code

[06:34:07] <stefandxm> once you are advancing in sql it will be more natural to do this

[06:34:33] <stefandxm> wow. thor movie stole the atlantis animation!

[06:34:49] <akiba__> SELECT count(1) FROM players WHERE player = '';

[06:35:11] <akiba__> if a >0 update (stats) else insert (stats)

[06:35:18] <akiba__> but you never specified a

[06:35:23] <stefandxm> a is a variable

[06:35:26] <stefandxm> you need to declare it somehow

[06:35:32] <akiba__> yes but you never declared it

[06:35:32] <stefandxm> and it depends on the dialect

[06:35:36] <stefandxm> nope

[06:35:42] <stefandxm> because i am not a mysql homie =)

[06:35:49] <akiba__> how do you declare that in the count?

[06:35:51] <akiba__> lol

[06:36:11] <stefandxm> i used mysql once

[06:36:16] <stefandxm> never more!

[06:36:34] <stefandxm> do you have a decent sql editor btw?

[06:36:36] <stefandxm> if not download one

[06:36:46] <stefandxm> so you can tweak your queries before you invoke them from java

[06:36:54] <stefandxm> http://dev.mysql.com/doc/refman/5.0/en/user-variables.html

[06:47:22] <akiba__> yeah dude

[06:47:35] <akiba__> you are talking about insert on duplicate

[06:48:13] <stefandxm> well updates, but yeah

[06:52:35] <akiba__> its called insert on duplicate

[06:52:39] <akiba__> it inserts it

[06:52:45] <akiba__> and if a duplicate is found

[06:52:49] <akiba__> it updates it

[06:56:12] <stefandxm> sounds odd =)

[06:59:37] <akiba__> lol

[06:59:45] <akiba__> also whats better

[06:59:50] <akiba__> flat file or database?

[06:59:52] <akiba__> for user accounts

[07:02:11] <stefandxm> database

[07:02:14] <stefandxm> always database

[07:02:19] <stefandxm> for non-media

[07:06:23] <akiba__> but flat file for media?

[07:07:15] <stefandxm> yeah

[07:07:39] <akiba__> so should i use mysql or nosql

[07:07:48] <stefandxm> not mysql

[07:07:50] <stefandxm> =D

[07:08:01] <stefandxm> but if your question is if you need transactions/sql .. depends ;)

[07:08:19] <akiba__> on what?

[07:08:41] <stefandxm> how you will deal with concurrency in your application

[07:10:07] <akiba__> what do u mean

[07:12:19] <stefandxm> mongodb and many other "no sql databases" will not have transactions

[07:12:54] <stefandxm> this means that you wont get an application lock on the data you are operating on, from the database. so you must implement it yourself

[07:13:02] <stefandxm> in a single threaded, single instance application this is trivial

[07:13:12] <stefandxm> in a highly parallell application it may not be that trivial

[07:13:39] <akiba__> like the mmo

[07:13:39] <stefandxm> so if you opt for say mongodb you will have to do some serious architecture/planning before you start writing your code

[07:13:51] <akiba__> i will probably have like 2000 people on at once

[07:13:59] <stefandxm> yes

[07:14:07] <akiba__> will sql handle that

[07:14:11] <stefandxm> and they will probably be fiddling around with same kind of data at the same time

[07:14:16] <stefandxm> 2000 is nothing

[07:14:35] <stefandxm> its all about how you manage your indexes/normalization in sql

[07:14:36] <akiba__> and its not the same kind of data currently

[07:14:47] <stefandxm> but its really the same pain in mongodb

[07:14:52] <akiba__> cause the way it is partitioned is separate files

[07:14:52] <stefandxm> just different :)

[07:14:58] <akiba__> 1 file for npcs

[07:15:03] <akiba__> 1 file for shops

[07:15:11] <akiba__> 1 file for npc spawn locartion

[07:15:32] <akiba__> 1 file for flagged accounts

[07:15:47] <akiba__> 1 file for ips/users banned/muted

[07:15:52] <akiba__> well 4 files total for that

[07:15:54] <akiba__> but 1 each

[07:16:11] <akiba__> 1 file for doors

[07:16:22] <akiba__> 1 file for objects

[07:16:30] <akiba__> 1 file for items

[07:16:40] <stefandxm> well

[07:16:46] <akiba__> 1 for prices

[07:16:47] <stefandxm> that means you would have to lock the files

[07:16:48] <stefandxm> individually

[07:17:05] <akiba__> and 2 for npc drops

[07:17:06] <stefandxm> becuase you will have multiple request in parallell

[07:17:13] <stefandxm> i'd say you should start with sql

[07:17:24] <stefandxm> since then you can easier get help

[07:17:29] <akiba__> equipment ids stored in 1 file

[07:17:40] <stefandxm> if you start with a non transactional database you need to know your shit from first line of code

[07:17:43] <akiba__> notes ids stored in another

[07:17:52] <akiba__> stackable ids stored in another

[07:18:46] <akiba__> 1 file for dupers

[07:18:57] <akiba__> 1 for worldmap

[07:19:00] <akiba__> and 1 for object sizes

[07:19:10] <akiba__> and idk how to code for nosql

[07:19:15] <akiba__> i just know its much faster

[07:19:32] <stefandxm> its not much faster if you havent a clever design to get around lockings

[07:19:46] <stefandxm> not in general

[07:19:49] <stefandxm> but often in practice ;)

[07:20:01] <akiba__> lol

[07:20:10] <akiba__> but if sql is the easiest

[07:20:17] <akiba__> is percona the best sql database?

[07:20:36] <stefandxm> never heard of it :o

[07:20:42] <stefandxm> sql is not easy

[07:20:50] <stefandxm> but sql is very well developed and there are alot of resources if you get stuck

[07:21:20] <stefandxm> tbh sql is a pain in the ass. the language is horrible for applications. you might want to use an ORM

[07:21:42] <stefandxm> but since you are quite new to stuff i dont want to recommend you to go all highlevel since you might not understand your own problems

[07:22:15] <akiba__> orm?

[07:22:17] <akiba__> what is that

[07:22:43] <akiba__> object oriented db?

[07:24:37] <stefandxm> http://en.wikipedia.org/wiki/Object-relational_mapping

[07:25:04] <stefandxm> you prety much get this "free" with mongodb drivers

[07:26:05] <akiba__> im thinking of this

[07:26:12] <akiba__> have a table only for reads

[07:26:19] <akiba__> that contains the servers config files

[07:26:51] <akiba__> ips and users banned should be i guess another table?

[07:27:10] <partycoder> wait

[07:27:20] <partycoder> what kind of configuration are you talking about

[07:27:23] <akiba__> a table for user accounts

[07:27:54] <akiba__> and a table for data like stackable ids etc

[07:28:09] <akiba__> and party im talking about switching my game server

[07:28:18] <akiba__> from flat file to database

[07:28:58] <partycoder> how do you make all game servers use the same configuration simultaneously

[07:29:04] <partycoder> or it's only 1

[07:29:17] <akiba__> currently i only have 1

[07:29:27] <akiba__> but majority of the data will be shared

[07:29:37] <partycoder> ok

[07:30:18] <akiba__> the npcs and shops might be different

[07:30:22] <akiba__> but everything else is the same

[07:32:03] <partycoder> well, mongodb for a simple configuration might be a bit overkill

[07:32:48] <partycoder> and well, you might want to have some sort of version control over things like that

[07:32:59] <partycoder> in case things go wrong

[07:34:32] <akiba__> its going to have a user account table as well

[07:34:49] <akiba__> which stores the users salted password

[07:34:58] <akiba__> well actually bcrypted password

[07:35:53] <akiba__> questioning if hashing the usernames is worth it

[07:36:00] <partycoder> not very recommended

[07:36:04] <partycoder> to encrypt passwords

[07:36:12] <partycoder> you usually use a 1 way hash

[07:36:28] <partycoder> and store the hash

[07:36:59] <partycoder> in that way you can verify a password not having to decrypt it

[07:37:00] <akiba__> 1 way hashes arent secure enough

[07:37:07] <akiba__> thats why they say use bcrypt

[07:37:07] <akiba__> or

[07:37:20] <partycoder> actually it should be more secure

[07:37:29] <partycoder> if you can decrypt a password...

[07:37:33] <akiba__> or pbkdf2

[07:37:38] <partycoder> in case of a data breach you are done

[07:37:51] <akiba__> and it would only take 100ms to compute the password

[07:38:08] <akiba__> vs hashing which is 10ms

[07:38:23] <akiba__> might be a small change

[07:38:38] <akiba__> but when you have to compute each individual combination

[07:38:43] <akiba__> until you get to the password

[07:38:47] <akiba__> it will take forever

[07:39:24] <partycoder> oh no

[07:40:01] <stefandxm> use a decent hashing mechanism

[07:40:04] <stefandxm> do not use md5

[07:40:08] <stefandxm> and use salt

[07:40:16] <partycoder> sha1 should be good enough

[07:40:21] <stefandxm> yeap

[07:40:23] <akiba__> sha1 isnt good enough

[07:40:27] <akiba__> sha256 isnt even good enough

[07:40:30] <partycoder> ok prove it

[07:41:25] <partycoder> most likely

[07:42:01] <stefandxm> ok

[07:42:11] <stefandxm> its hard to tell when so many are on same hosts

[07:42:17] <sellout> akiba__: But bcrypt _is_ a 1-way hash.

[07:42:47] <stefandxm> sha1 is often used > 4000 times

[07:42:49] <stefandxm> to be considered safe

[07:43:14] <akiba__> thats why i was thinking about

[07:43:18] <stefandxm> speaking of wich.. i should resume my sasl scram coding =)

[07:43:22] <akiba__> sha256ing usernames and passwords

[07:43:31] <akiba__> or sha1ing usernames and sha256ing passwords

[07:43:37] <akiba__> so even if they break in

[07:43:43] <akiba__> they have no clue what to do

[07:43:50] <partycoder> ehm

[07:43:58] <partycoder> that's security by obscurity

[07:44:00] <stefandxm> if they break in

[07:44:09] <partycoder> and they might figure it out from the client/server protocol

[07:44:13] <stefandxm> and you have your own authentication mechanisms

[07:44:16] <stefandxm> then they can just spoof

[07:44:24] <akiba__> sellout: bcrypt would work normally wouldnt it?

[07:44:27] <stefandxm> only way to get this safe is to get the encryption done in the browser

[07:44:35] <stefandxm> and never send the password to you

[07:44:36] <sellout> akiba__: bcrypt is the right thing for passwords.

[07:45:05] <akiba__> thought so

[07:45:12] <akiba__> so how would i use it in java?

[07:46:05] <sellout> I could see hashing usernames for obscurity (like, making it harder for a hacker to figure out who the users on your porn site are), but if you’re storing billing info, or full names / display names or other identifiable stuff, it’s not helpful.

[07:46:42] <akiba__> yeah sellout ik

[07:46:57] <akiba__> the character files dont store much though

[07:47:13] <akiba__> the username the password possibly an email

[07:47:18] <akiba__> and the character stats

[07:47:27] <akiba__> character stats arent identifiable

[07:47:36] <akiba__> email is only used for password recovery

[07:47:55] <akiba__> and names+passwords can be encrypted

[07:48:04] <akiba__> well names hashed and passwds encrypted

[07:48:11] <akiba__> so they cant find anything

[07:48:16] <sellout> And, you can’t salt them (so you can’t use bcrypt), because then you couldn’t look them up, and dictionary attacks are probably pretty effective against usernames, so unsalted isn’t too useful either.

[07:48:26] <partycoder> there's a difference between hashing and encryption

[07:48:32] <sellout> But, I haven’t really thought about it at all, so *shrug*

[07:48:41] <stefandxm> again

[07:48:46] <stefandxm> if they hack your backend server

[07:48:46] <partycoder> with encryption you can decrypt

[07:48:47] <sellout> akiba__: Don’t encrypt passwords.

[07:48:52] <akiba__> you cant salt passwords in java?

[07:48:54] <stefandxm> what stops them from getting the plain username+password?

[07:48:57] <stefandxm> directly from a login

[07:49:01] <partycoder> and you don't want to ever have the possibility to decrypt

[07:49:08] <sellout> akiba__: No, you can’t salt usernames.

[07:49:53] <akiba__> what if i found the hash in sha 256

[07:49:58] <akiba__> and then salted that

[07:50:01] <akiba__> would that be good enough?

[07:50:13] <sellout> akiba__: What?

[07:50:40] <akiba__> hash the passwords with sha256

[07:50:45] <akiba__> and then after hashing them salt them

[07:50:55] <sellout> akiba__: Hash the passwords with bcrypt.

[07:50:56] <akiba__> cause you are saying bcrypt wouldnt work

[07:51:06] <stefandxm> noone said bcrypt wont work

[07:51:34] <sellout> akiba__: I’m saying you can’t use bcrypt (which requires a salt) if you want to hash _usernames_.

[07:51:45] <sellout> You should use bcrypt for passwords.

[07:51:54] <akiba__> so bcrypt for passwords

[07:52:02] <akiba__> and a regular sha256 for usernames?

[07:52:30] <sellout> Well, I don’t think hashing usernames is going to buy you anything.

[07:52:40] <akiba__> most of the hackers in this community

[07:52:46] <akiba__> dont really know what they are doing lol

[07:52:53] <akiba__> and you know the fun part :P

[07:53:01] <sellout> Because you can’t salt them, and they’re dictionary attackable, and you’re storing an email address.

[07:53:02] <stefandxm> actually most "hackers" are just bots

[07:53:03] <akiba__> i dont see how they are getting into the database

[07:53:15] <stefandxm> bots+cheap labour from the east

[07:53:21] <akiba__> my websites are behind cloudflare

[07:53:33] <akiba__> *implies china* lol

[07:53:39] <stefandxm> akiba__, the easiest way for them would be by attacking your sql queries that are concentated

[07:53:52] <akiba__> would that work though?

[07:53:56] <sellout> stefandxm: :D

[07:53:58] <stefandxm> yes it would

[07:54:03] <akiba__> the web server is on a different server from the game server

[07:54:07] <stefandxm> doesnt matter

[07:54:22] <akiba__> and the game server only answers responses that are from the uid

[07:55:07] <sellout> http://xkcd.com/327/ – relevant

[07:55:07] <akiba__> and why wouldnt it matter if they are different servers?

[07:55:21] <akiba__> my web server/database ip isnt exposed at all

[07:55:24] <stefandxm> little bobby tables!! =)

[07:55:32] <sellout> Anyway, sleep now. Have fun, akiba__

[07:55:32] <akiba__> the only thing exposed is the game server

[07:55:43] <akiba__> and they are on completely different networks

[07:55:48] <akiba__> and good night sellbot

[07:55:58] <akiba__> sellout*

[07:55:58] <akiba__> lol

[07:56:38] <akiba__> but yeah stefandxm: thats not good enough to protect my server?

[07:56:50] <akiba__> hiding database/web server behind cloudflare

[07:56:51] <stefandxm> no

[07:56:55] <stefandxm> you need to sanitize your code

[07:57:05] <akiba__> and completely different networks for game server and web server?

[07:57:10] <stefandxm> doesnt matter

[07:57:19] <stefandxm> since you are allowing people to enter data to the database somehow right?

[07:57:22] <stefandxm> or by just name

[07:57:27] <stefandxm> as you see in xkcd strip above

[07:57:33] <sellout> akiba__: “completely different networks” implies that those two servers can’t talk to each other ;)

[07:57:36] <stefandxm> its enough to pipe your entire database to russia

[07:57:37] <sellout> ok, goodnight :D

[07:57:40] <akiba__> sellout yeah they can

[07:57:46] <akiba__> they are on completely different networks

[07:57:50] <akiba__> but they would get ssh tunneled

[07:58:40] <akiba__> LOL

[07:59:35] <akiba__> ok guys

[07:59:40] <akiba__> so how do i make a query

[07:59:46] <akiba__> that will stop that from happening

[07:59:54] <stefandxm> use prepared statements

[07:59:59] <stefandxm> and dont concentate any data

[08:00:08] <stefandxm> use parameters as we talked about before

[08:00:45] <partycoder> ehm

[08:00:57] <akiba__> ok so thats 1 script done

[08:00:59] <akiba__> now lets see

[08:02:30] <akiba__> what about this

[08:02:31] <akiba__> public static void ping(){ try { String query = "SELECT * FROM donation WHERE username = 'null'"; query(query); } catch (Exception e) { e.printStackTrace(); con = null; stm = null; } }

[08:03:23] <partycoder> that's not mongo

[08:03:55] <stefandxm> akiba__, you praobably mean null not 'null'

[08:04:11] <stefandxm> other than that theres no external data going into the query so it should be safe

[08:04:23] <akiba__> whats the diff between null and 'null'?

[08:04:54] <stefandxm> 'null' is a literal string null is .. null :)

[08:05:13] <stefandxm> http://en.wikipedia.org/wiki/Null_%28SQL%29

[08:05:18] <akiba__> lol

[08:05:32] <stefandxm> null is "doesnt exist"

[08:05:44] <stefandxm> and any logical statement including it will be false or null

[08:06:03] <akiba__> alright heres a good one

[08:06:27] <akiba__> http://pastie.org/9336956

[08:10:16] <akiba__> http://pastie.org/9336965

[08:11:05] <akiba__> http://pastie.org/9336967

[08:11:13] <akiba__> they dont look good at all :P

[08:20:37] <akiba__> stefandxm are you still reading the code?

[08:56:23] <stefandxm> akiba__, sorry no

[08:56:34] <stefandxm> akiba__, daytime here now. too much other stuff to do :)

[08:57:21] <akiba__> haha ok

[08:57:25] <akiba__> and what time is it there?

[09:16:18] <akiba__> hey guys

[09:16:24] <akiba__> anyone know a java programmer i could use?

[12:03:05] <AliRezaTaleghani> Hi all

[12:03:39] <AliRezaTaleghani> I'm facing with a problem on my production env

[12:03:52] <AliRezaTaleghani> we deployed a TAG based shard

[12:04:11] <AliRezaTaleghani> to store some logs

[12:04:33] <AliRezaTaleghani> all logs have a field dateCreated which is a ISODate

[12:04:58] <AliRezaTaleghani> and shards TAGs are based on this field too

[12:05:19] <AliRezaTaleghani> but new logs are inserting on wrong shard...

[12:06:57] <AliRezaTaleghani> it's my shard config

[12:06:59] <AliRezaTaleghani> http://pastie.org/9337314

[12:07:51] <AliRezaTaleghani> :-/ but but I have lot's on today logs on Archive shard0001

[12:11:23] <joannac> where's the maxchunk?

[12:12:09] <AliRezaTaleghani> joannac: are u asking me?

[12:12:09] <AliRezaTaleghani> about my problem :-/

[12:12:14] <joannac> yes

[12:12:25] <joannac> also, why are you using a monotonically increasing shard key?

[12:17:47] <AliRezaTaleghani> joannac: I can't find what is the value of current maxChunk :-/

[12:17:55] <AliRezaTaleghani> where I s should look forward?

[12:19:41] <AliRezaTaleghani> joannac: I thinks it's current value is 64

[12:19:42] <AliRezaTaleghani> http://pastie.org/9337351

[12:19:59] <joannac> nooo

[12:20:11] <joannac> in your shell connected to a mongos

[12:20:17] <AliRezaTaleghani> and about the "monotonicllty increaing shard key" let me search to understant what exacllty you mean

[12:20:21] <AliRezaTaleghani> ok

[12:20:30] <AliRezaTaleghani> I'm on the sheel right now

[12:20:33] <AliRezaTaleghani> shell*

[12:21:25] <AliRezaTaleghani> joannac: and...? :-?

[12:21:59] <joannac> use config; db.chunks.find({"max.dateCreated": MaxKey})

[12:22:37] <joannac> http://docs.mongodb.org/manual/core/sharding-shard-key/#write-scaling

[12:23:46] <AliRezaTaleghani> http://pastie.org/9337362

[12:24:30] <joannac> yup, your maximum chunk is on shard0001

[12:24:38] <joannac> and so all your writes are going there

[12:24:52] <AliRezaTaleghani> joannac: about the monotonically key, it's cos need to have a Tier2 (Archive model) as fallow

[12:24:53] <AliRezaTaleghani> http://blog.mongodb.org/post/85721044164/tiered-storage-models-in-mongodb-optimizing-latency

[12:25:25] <AliRezaTaleghani> joannac: and what dose it mean? who can I switch maximum chunk on shard0001?

[12:25:51] <joannac> oh i see. makes sense

[12:26:14] <joannac> it means until you move your data, all your writes will go to the max chunk

[12:26:45] <AliRezaTaleghani> the problem is that the current new logs which are for current day! I think should be store on shard0000... but :-/

[12:27:09] <joannac> you could try moveChunk

[12:27:10] <joannac> http://docs.mongodb.org/manual/reference/command/moveChunk/

[12:27:17] <joannac> but you need to turn off the balancer

[12:28:39] <AliRezaTaleghani> I had stopped Balance right now

[12:29:42] <AliRezaTaleghani> but maybe cos of the running moveChunk process the Archive disks are fully load

[12:30:23] <AliRezaTaleghani> do u mean I should move the last chunk to shard000?

[12:30:27] <AliRezaTaleghani> manually

[12:32:19] <joannac> yes, moveChunk the max chunk to shard000

[12:32:41] <joannac> then all your inserts for today will to go shard000 instead

[12:33:04] <AliRezaTaleghani> but how about the next inserts?

[12:33:11] <AliRezaTaleghani> for example tomorrow?!

[12:33:13] <joannac> what next inserts?

[12:33:24] <joannac> your maximum chunk will be on shard000

[12:33:32] <AliRezaTaleghani> I need In real time I have inserting logs....

[12:33:33] <AliRezaTaleghani> ok

[12:33:34] <joannac> all inserts will go there

[12:33:41] <AliRezaTaleghani> should this shoud be permanent?

[12:33:47] <joannac> because your shard key is always increasing

[12:33:47] <AliRezaTaleghani> ok

[12:33:52] <AliRezaTaleghani> let me check it...

[12:34:06] <AliRezaTaleghani> on production env! :D

[12:34:17] <joannac> good luck

[12:34:42] <AliRezaTaleghani> joannac: :-) thanks...

[12:34:47] <AliRezaTaleghani> I hope too

[12:35:13] <AliRezaTaleghani> this mean that my max_key shard was defined incorrect from the first....

[12:35:15] <AliRezaTaleghani> yea?

[12:40:30] <LesTR> sure, it's realy bad idea based shard key on permanent increasing value

[12:41:23] <LesTR> your setup use only one shard in one time (for write)

[12:43:32] <AliRezaTaleghani> joannac: :"> for the last: those this mean:

[12:43:32] <AliRezaTaleghani> sh.moveChunk("aban-sdp.event", {"min" : {"dateCreated" : ISODate("2014-06-29T12:16:30.889Z")}} , "shard0001")

[12:44:01] <AliRezaTaleghani> I wrote this regard to the maxchunk which you learned me to find

[12:51:39] <AliRezaTaleghani> joannac: :D I'm feel myself just on a puzzel

[12:51:39] <AliRezaTaleghani> http://pastie.org/9337431

[12:52:37] <AliRezaTaleghani> maybe I should start balance and let it to move it self on a safe position :-/

[14:20:12] <salty-horse> hey. I moved my sharded database to a different address. it there a way to tell mongoconf that the shards have a new address? I see commands to create new empty shards, but I don't thin that's appropriate

[18:32:31] <richthegeek_> I have a system which uses multiple databases (multitenant so one per account). How is it recommended to do connections in such a system? At the moment I have one per database (literally creating a new MongoDB.Server and MongoDB.DB using that server)

[18:32:52] <richthegeek_> but it's possible to both reuse the server, and to call db.getSiblingDb to get a database

[18:33:00] <richthegeek_> should I be doing that? or does it generally make no difference?

[18:34:36] <richthegeek_> in MMS it's listing ~140 connections (14 databases, minimum of 3 collections and typically ~10 or so used per db) - would reducing the number reduce memory usage or alter performance?

[18:38:11] <richthegeek_> well just tried it getting a "a server cant be shared across multiple Db instances" so thats simple enough

[18:40:59] <richthegeek_> so i guess that has me wondering: why do i have 140 connections when I have only 14 databases and 2 processes connected (plus 2 repl sets)

[19:02:46] <richthegeek_> ok, changed to using a single MongoClient per host/port combo (yay bluebird) and now i have only 31 active connections :D

[20:39:15] <noizex> hi, anyone tried compiling latest legacy mongo c++ driver source on windows using cl.exe?

[20:43:49] <Zelest> Derick, is there any known issues I could run in two if I run a replicaset with mixed versions? e.g, 2.4.10 and 2.6.3 ?

[20:44:09] <Zelest> I assume it's not recommended, but is it posisble?

[20:50:45] <noizex> weird, it looks like that mongoclient source misses a lot of header includes :/

[20:51:14] <noizex> I wonder if anyone even tried to compile it on windows ;)

[21:22:48] <Peavey_> Does anyone know of any settings to reduce the startup time for MongoDB in terms of opening connections to all of its DB files?

Log file Viewer

Help | Karma | Search:

#mongodb logs for Sunday the 29th of June, 2014