java - mapreduce task running only on master -


i trying run mapreduce on 2 node cluster using hadoop 2.6. mapreduce running on master node not on slave node.

2015-05-18 14:46:43,570 info org.apache.hadoop.yarn.server.nodemanager.nodemanager: registered unix signal handlers [term, hup, int] 2015-05-18 14:46:44,311 info org.apache.hadoop.yarn.event.asyncdispatcher: registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.container.containereventtype class org.apache.hadoop.yarn.server.nodemanager.containermanager.containermanagerimpl$containereventdispatcher 2015-05-18 14:46:44,311 info org.apache.hadoop.yarn.event.asyncdispatcher: registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.application.applicationeventtype class org.apache.hadoop.yarn.server.nodemanager.containermanager.containermanagerimpl$applicationeventdispatcher 2015-05-18 14:46:44,311 info org.apache.hadoop.yarn.event.asyncdispatcher: registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.localizationeventtype class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.resourcelocalizationservice 2015-05-18 14:46:44,312 info org.apache.hadoop.yarn.event.asyncdispatcher: registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.auxserviceseventtype class org.apache.hadoop.yarn.server.nodemanager.containermanager.auxservices 2015-05-18 14:46:44,312 info org.apache.hadoop.yarn.event.asyncdispatcher: registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.containersmonitoreventtype class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.containersmonitorimpl 2015-05-18 14:46:44,312 info org.apache.hadoop.yarn.event.asyncdispatcher: registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.containerslaunchereventtype class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.containerslauncher 2015-05-18 14:46:44,325 info org.apache.hadoop.yarn.event.asyncdispatcher: registering class org.apache.hadoop.yarn.server.nodemanager.containermanagereventtype class org.apache.hadoop.yarn.server.nodemanager.containermanager.containermanagerimpl 2015-05-18 14:46:44,325 info org.apache.hadoop.yarn.event.asyncdispatcher: registering class org.apache.hadoop.yarn.server.nodemanager.nodemanagereventtype class org.apache.hadoop.yarn.server.nodemanager.nodemanager 2015-05-18 14:46:44,348 info org.apache.hadoop.metrics2.impl.metricsconfig: loaded properties hadoop-metrics2.properties 2015-05-18 14:46:44,388 info org.apache.hadoop.metrics2.impl.metricssystemimpl: scheduled snapshot period @ 10 second(s). 2015-05-18 14:46:44,388 info org.apache.hadoop.metrics2.impl.metricssystemimpl: nodemanager metrics system started 2015-05-18 14:46:44,402 info org.apache.hadoop.yarn.event.asyncdispatcher: registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.loghandlereventtype class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.nonaggregatingloghandler 2015-05-18 14:46:44,402 info org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.resourcelocalizationservice: per directory file limit = 8192 2015-05-18 14:46:44,419 info org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.resourcelocalizationservice: usercache path : file:/tmp/hadoop-hduser/nm-local-dir/usercache_del_1431940604404 2015-05-18 14:46:44,446 info org.apache.hadoop.yarn.event.asyncdispatcher: registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.localizereventtype class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.resourcelocalizationservice$localizertracker 2015-05-18 14:46:44,467 warn org.apache.hadoop.yarn.server.nodemanager.containermanager.auxservices: auxilurary service named 'mapreduce_shuffle' in configuration class class org.apache.hadoop.mapred.shufflehandler has name of 'httpshuffle'. because these not same tools trying send servicedata , read service meta data may have issues unless refer name in config. 2015-05-18 14:46:44,467 info org.apache.hadoop.yarn.server.nodemanager.containermanager.auxservices: adding auxiliary service httpshuffle, "mapreduce_shuffle" 2015-05-18 14:46:44,489 info org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.containersmonitorimpl:  using resourcecalculatorplugin : org.apache.hadoop.yarn.util.linuxresourcecalculatorplugin@8662911 2015-05-18 14:46:44,489 info org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.containersmonitorimpl:  using resourcecalculatorprocesstree : null 2015-05-18 14:46:44,489 info org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.containersmonitorimpl: physical memory check enabled: true 2015-05-18 14:46:44,489 info org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.containersmonitorimpl: virtual memory check enabled: true 2015-05-18 14:46:44,491 warn org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.containersmonitorimpl: nodemanager configured 8 g physical memory allocated containers, more 80% of total physical memory available (3.8 g). thrashing might happen. 2015-05-18 14:46:44,494 info org.apache.hadoop.yarn.server.nodemanager.nodestatusupdaterimpl: initialized nodemanager null: physical-memory=8192 virtual-memory=17204 virtual-cores=8 2015-05-18 14:46:44,588 info org.apache.hadoop.ipc.callqueuemanager: using callqueue class java.util.concurrent.linkedblockingqueue 2015-05-18 14:46:44,648 info org.apache.hadoop.ipc.server: starting socket reader #1 port 48733 2015-05-18 14:46:44,785 info org.apache.hadoop.yarn.factories.impl.pb.rpcserverfactorypbimpl: adding protocol org.apache.hadoop.yarn.api.containermanagementprotocolpb server 2015-05-18 14:46:44,785 info org.apache.hadoop.yarn.server.nodemanager.containermanager.containermanagerimpl: blocking new container-requests container manager rpc server still starting. 2015-05-18 14:46:44,786 info org.apache.hadoop.ipc.server: ipc server responder: starting 2015-05-18 14:46:44,786 info org.apache.hadoop.ipc.server: ipc server listener on 48733: starting 2015-05-18 14:46:44,805 info org.apache.hadoop.yarn.server.nodemanager.security.nmcontainertokensecretmanager: updating node address : egap-optiplex-3020:48733 2015-05-18 14:46:44,819 info org.apache.hadoop.ipc.callqueuemanager: using callqueue class java.util.concurrent.linkedblockingqueue 2015-05-18 14:46:44,819 info org.apache.hadoop.ipc.server: starting socket reader #1 port 8040 2015-05-18 14:46:44,821 info org.apache.hadoop.yarn.factories.impl.pb.rpcserverfactorypbimpl: adding protocol org.apache.hadoop.yarn.server.nodemanager.api.localizationprotocolpb server 2015-05-18 14:46:44,822 info org.apache.hadoop.ipc.server: ipc server responder: starting 2015-05-18 14:46:44,822 info org.apache.hadoop.ipc.server: ipc server listener on 8040: starting 2015-05-18 14:46:44,822 info org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.resourcelocalizationservice: localizer started on port 8040 2015-05-18 14:46:44,867 info org.apache.hadoop.mapred.indexcache: indexcache created max memory = 10485760 2015-05-18 14:46:44,876 info org.apache.hadoop.mapred.shufflehandler: httpshuffle listening on port 13562 2015-05-18 14:46:44,877 info org.apache.hadoop.yarn.server.nodemanager.containermanager.containermanagerimpl: containermanager started @ egap-optiplex-3020/127.0.1.1:48733 2015-05-18 14:46:44,877 info org.apache.hadoop.yarn.server.nodemanager.containermanager.containermanagerimpl: containermanager bound 0.0.0.0/0.0.0.0:0 2015-05-18 14:46:44,882 info org.apache.hadoop.yarn.server.nodemanager.webapp.webserver: instantiating nmwebapp @ 0.0.0.0:8042 2015-05-18 14:46:44,981 info org.mortbay.log: logging org.slf4j.impl.log4jloggeradapter(org.mortbay.log) via org.mortbay.log.slf4jlog 2015-05-18 14:46:44,983 info org.apache.hadoop.http.httprequestlog: http request log http.requests.nodemanager not defined 2015-05-18 14:46:44,993 info org.apache.hadoop.http.httpserver2: added global filter 'safety' (class=org.apache.hadoop.http.httpserver2$quotinginputfilter) 2015-05-18 14:46:44,994 info org.apache.hadoop.http.httpserver2: added filter static_user_filter (class=org.apache.hadoop.http.lib.staticuserwebfilter$staticuserfilter) context node 2015-05-18 14:46:44,994 info org.apache.hadoop.http.httpserver2: added filter static_user_filter (class=org.apache.hadoop.http.lib.staticuserwebfilter$staticuserfilter) context static 2015-05-18 14:46:44,994 info org.apache.hadoop.http.httpserver2: added filter static_user_filter (class=org.apache.hadoop.http.lib.staticuserwebfilter$staticuserfilter) context logs 2015-05-18 14:46:44,997 info org.apache.hadoop.http.httpserver2: adding path spec: /node/* 2015-05-18 14:46:44,997 info org.apache.hadoop.http.httpserver2: adding path spec: /ws/* 2015-05-18 14:46:45,019 info org.apache.hadoop.http.httpserver2: jetty bound port 8042 2015-05-18 14:46:45,019 info org.mortbay.log: jetty-6.1.26 2015-05-18 14:46:45,051 info org.mortbay.log: extract jar:file:/usr/local/hadoop/share/hadoop/yarn/hadoop-yarn-common-2.6.0.jar!/webapps/node /tmp/jetty_0_0_0_0_8042_node____19tj0x/webapp 2015-05-18 14:46:45,300 info org.mortbay.log: started httpserver2$selectchannelconnectorwithsafestartup@0.0.0.0:8042 2015-05-18 14:46:45,300 info org.apache.hadoop.yarn.webapp.webapps: web app /node started @ 8042 2015-05-18 14:46:45,607 info org.apache.hadoop.yarn.webapp.webapps: registered webapp guice modules 2015-05-18 14:46:45,646 info org.apache.hadoop.yarn.client.rmproxy: connecting resourcemanager @ /10.182.1.29:8031 2015-05-18 14:46:45,713 info org.apache.hadoop.yarn.server.nodemanager.nodestatusupdaterimpl: sending out 0 nm container statuses: [] 2015-05-18 14:46:45,737 info org.apache.hadoop.yarn.server.nodemanager.nodestatusupdaterimpl: registering rm using containers :[] 2015-05-18 14:46:45,924 info org.apache.hadoop.yarn.server.nodemanager.security.nmcontainertokensecretmanager: rolling master-key container-tokens, got key id -185600296 2015-05-18 14:46:45,926 info org.apache.hadoop.yarn.server.nodemanager.security.nmtokensecretmanagerinnm: rolling master-key container-tokens, got key id -442584774 2015-05-18 14:46:45,926 info org.apache.hadoop.yarn.server.nodemanager.nodestatusupdaterimpl: registered resourcemanager egap-optiplex-3020:48733 total resource of <memory:8192, vcores:8> 2015-05-18 14:46:45,926 info org.apache.hadoop.yarn.server.nodemanager.nodestatusupdaterimpl: notifying containermanager unblock new container-requests

this complete nodemanager log on slave node.i don’t think error message here.


Comments

Post a Comment

Popular posts from this blog

c++ - Difference between pre and post decrement in recursive function argument -

i have following sample code used pre-decrement void function(int c) { if(c == 0) { return; } else { cout << "dp" << c << endl; function(--c); cout << c << endl; return; } } this function give output input 4 : dp3 dp2 dp1 dp0 0 1 2 3 but when use post decrement void function(int c) { if(c == 0) { return; } else { cout << "dp" << c << endl; function(c--); cout << c << endl; return; } } output goes in infinite loop dp4 can explain me in detail why happens ? this happens because function(c--) called same value of c , when finishes c decremented. recursively called, hence called same value no chance of return , hit stack overflow error. assume this, int x = 1, y = 0; y = x++ result y == 1 , x == 2. if y = ++x , bot x , y 2.
Read more

php - Nothing but 'run(); ' when browsing to my local project, how do I fix this? -

i added new project. made mistake in projectname.conf file in sites-available apache2, led me reinstalling apache2. after reinstalling , re-enabling projects on every page of every project .. <body> run(); </body> .. i've run composer update, restarted service, restarted pc, checked config files, tried disabling new project. there nothing in error log, , access.log: 127.0.0.1 - - [15/may/2015:01:40:47 +0200] "get / http/1.1" 304 181 "-" "mozilla/5.0 (x11; ubuntu; linux x86_64; rv:38.0) gecko/20100101 firefox/38.0" maybe i'm forgetting apache2 reinstall? don't think did different last time installed it. looks haven't setup php correctly. take @ this great guide , maybe you're forgetting link php config apache2.
Read more

php - How can I echo out this array? -

i don't know how go grabbing data in variable , put foreach loop. class tree{ private $info = array( array('name'=>'jim','state'=>'nu','company'=>'nu','phone'=>array('cell'=>'5615111111','office'=>'5611111111'),'email'=>array('primary'=>'exs@example.com','ex@there.com')), array('name'=>'joe smith','city'=>'phoenix','phone'=>'4805551111','email'=>'jsmith@some_email.com'), array('name'=>'john doe','city'=>'chandler','company'=>'doe co.','email'=>array('jdoe@gmail.com','personal'=>'email@email.com'),'phone'=>'6025550002') ); } if you're inside of class, can access private variable using $this->variablename . example, i...
Read more