elasticsearch - Cost effective and timely system to do Business Logic queries (batch processing) on weekly snapshots of a 15 gb MySQL database? -


we want batch operations on series of mysql snapshots, unsure of architecture purseu. have on 100 snapshots of 15 gb database.

we debating architecture use wouldn't embarrassing slow, not huge overhead (time-wise or monetarily). here have considered:

loading each db open stack vm doing our query saving data found off somewhere in redis instance can use python dump reports. (this slow no?)

loading elastic search , perform queries via elastic search (this not allow robustness/logic of querying want.)

we have looked @ hive or prestodb require large amounts of hardware?

on small scale of our tables this:

    client:         client_id (primary key)         name         address         city      device:         device_id (primary key)         name         location      service:         service_id (primary key)         name         price         client_id         device_id      (there no relationships between tables) 

here example of want do:

look @ services have increased in price on last month of snapshots (4 snapshots). client's location of client , device name each service increased. dump csv or other visualizing software kibana.


Comments

Popular posts from this blog

c++ - Difference between pre and post decrement in recursive function argument -

php - Nothing but 'run(); ' when browsing to my local project, how do I fix this? -

php - How can I echo out this array? -