Relational database versus R/Python data frames -


i exposed world of tables , data structures in r before rdbms systems , other database systems. quite elegant in r/python create tables , lists stuctured data (.csv or other formats) , data manipulations programmatically.

last year, attended course in database management , learnt structured , unstructured databases. noticed norm feed data multiple sources of data databases rather directly use them in r (for convenience , discipline?).

for research purposes, r seems suffice, joining, appending or complicated data manipulations.

the questions keeps arising is: when use r directly using commands such read.csv, when use r creating database , querying tables using r-sql interface?

for instance, if have multi-source data, (a) person level information (age, gender, smoking habits), (b) outcome variables (such surveys taken them in real time), (c) covariate information (environment characteristics), (d) treatment input (occurrence of event modifies outcome - survey response) (d) time , space information of participants taking survey

how approach data collection , processing in case. there may standard industry procedures, put question forward here, understand list of feasible , optimal approaches individuals , small group of researchers can adopt.

what you're describing when "that norm feed data multiple sources of data databases" sounds more data warehouse. databases used many reasons, , in plenty of situations hold data 1 source - instance, database used data store of transactional system hold data needed run system, , data produced system.

the process you're describing commonly called extract, transform, load (etl), , might find looking information etl , data warehousing helpful if decide go in direction of combining data prior working in r.

i can't tell should choose, or optimal way of accomplishing it, because vary in different situations , might come down opinion. can tell of reasons why people create data warehouses, , can decide whether might useful in situation:

a data warehouse can provide central location hold combined data. means people not need combine data each time need use specific combination of data. unlike simple one-off report or extract of combined data, should provide flexibility, letting people obtain combined set of data need specific task. often, in enterprise situations, multiple things run on top of same combined set of data - multidimensional data analysis tools (cubes), reports, data mining, etc.

some of benefits of might include:

  • individuals saving time when otherwise have needed combine data themselves.
  • if data needs combined complex, or people not have proficiency @ handling part of process, there less risk of data being combined incorrectly; can sure different pieces of work have used same source data.
  • if data suffers data quality issues, resolve once in data warehouse, rather working around or resolving repeatedly in code.
  • if new data being received, collection , integration of data warehouse can carried out automatically.

like say, can't decide whether useful direction or not - decision of kind you'll need weigh costs of implementing such solution against benefits, , both specific individual case. answers core question of why might choose work in database instead of in code, , gives starting point work from.


Comments

Post a Comment

Popular posts from this blog

Email notification in google apps script -

https://support.google.com/docs/answer/91588?hl=en i have sheet the cells filled in data pulled via apps script, website. pull in exact same data, not. on old sheets version use able emails when data "changes" different value. new sheets, email when new data pasted cells, though exact same data......and, can't set @ ranges anymore, entire document. there easy way watch cells when data changes, , email myself? this looks lot you're trying do. for checking whether value has changed after edit, this gives necessary information it. hence, know whether new value same 1 or not. implementing "workaround" talked in second link, able compare , send email myself if value changed. initiate onedit trigger goes off onedit function uses these lines after checks in place have passed. var email = "xyz@whatever.com"; mailapp.sendemail(email, "triggermail", "a change has occurred in sheet"); note: sends "notification...
Read more

c++ - Difference between pre and post decrement in recursive function argument -

i have following sample code used pre-decrement void function(int c) { if(c == 0) { return; } else { cout << "dp" << c << endl; function(--c); cout << c << endl; return; } } this function give output input 4 : dp3 dp2 dp1 dp0 0 1 2 3 but when use post decrement void function(int c) { if(c == 0) { return; } else { cout << "dp" << c << endl; function(c--); cout << c << endl; return; } } output goes in infinite loop dp4 can explain me in detail why happens ? this happens because function(c--) called same value of c , when finishes c decremented. recursively called, hence called same value no chance of return , hit stack overflow error. assume this, int x = 1, y = 0; y = x++ result y == 1 , x == 2. if y = ++x , bot x , y 2.
Read more

javascript - IE11 incompatibility with jQuery's 'readonly'? -

the code i'm running has no issues in firefox, or chrome. user typing 1 textbox, tabs next , can continue typing that. when run webpage on ie11, user tabs next textbox , can't type in. textbox has double-clicked before user can insert anything. i've googled , seems ie give problems when using older jquery (i.e. prefers 'prop' 'attr' ). in code below jquery's readonly keyword being used in way ie11 no longer recognise/accept? i assume issue readonly , removeclass('ignore') working expected. //when focus on textbox 1 $("#field1").focus(function() { //remove readonly property textbox 1 $('#field1').prop('readonly', false).removeclass('ignore'); //add readonly property other textboxes $('#field2, #field3, #field4') .prop('readonly', 'readonly').addclass('ignore').val(''); }); i'm using jquery 2.1.3 ...
Read more