java - Running Trident Topology in Storm TrackedTopology Unit Test -


how can run junit test of trident topology allow tuples flow through topology while testing , verifying output @ each stage? i've tried running within storm's testing framework, it's falling short of allowing verification , consistent execution of trident.

here's example topology in-line comments i'm having issues.

import static org.junit.assert.assertequals; import java.util.arrays; import java.util.list; import org.junit.test; import storm.trident.tridentstate; import storm.trident.tridenttopology; import storm.trident.operation.builtin.count; import storm.trident.testing.memorymapstate; import storm.trident.testing.split; import backtype.storm.config; import backtype.storm.ilocalcluster; import backtype.storm.testing; import backtype.storm.testing.feederspout; import backtype.storm.testing.testjob; import backtype.storm.testing.trackedtopology; import backtype.storm.tuple.fields; import backtype.storm.utils.utils;  public class wordcounttopologytest {      @test     public void testwordcounttopology() throws exception {         testing.withtrackedcluster(new wordcounttestjob());     }      public class wordcounttestjob implements testjob {          @override         public void run(ilocalcluster cluster) throws exception {              // create test topology submit             tridenttopology termcounttopology = new tridenttopology();              feederspout feeder = new feederspout(new fields("text", "author"));              tridentstate tridentstate = termcounttopology.newstream("inputspout", feeder)                     .each(new fields("text"), new split(), new fields("word"))                    .groupby(new fields("word"))                    .name("counter-output")                    .persistentaggregate(new memorymapstate.factory(), new count(), new fields("count"))                                    .parallelismhint(6);              trackedtopology tracked = testing.mktrackedtopology(cluster, termcounttopology.build());              // feed random data topology             feeder.feed(arrays.aslist("nearly men can stand adversity, if want test man's character, give him power.", "abraham lincoln"));             feeder.feed(arrays.aslist("no man has enough memory successful liar.", "abraham lincoln"));             feeder.feed(arrays.aslist("either write worth reading or worth writing.", "benjamin franklin"));             feeder.feed(arrays.aslist("well done better said.", "benjamin franklin"));              cluster.submittopology("word-count-testing", new config(), tracked.gettopology());              // (!!) runs, bad sleep time when may run faster or slower on other systems             // utils.sleep(5000);              // (!!) fails 5000ms topology timeout             // testing.trackedwait(tracked, 3);              /*              * (!!) 0. trident creates streams , bolts internally              * different names, how can read them verify?              */             list outputtuples = testing.readtuples(tracked, "counter-output");             assertequals(0, outputtuples.size());         }     } }

beyond this, i've tried writing own basefilter tag on end of stream stores of tuples, seems there must better way. also, doesn't solve issue of running topology in controlled manner. trident supports?

use class feederbatchspout (for trident) instead of feederspout. feederbatchspout blocks itself, there no need use testing.trackedwait() or that.

source: https://groups.google.com/forum/#!topic/storm-user/cradqexo5ou


Comments

Post a Comment

Popular posts from this blog

Email notification in google apps script -

https://support.google.com/docs/answer/91588?hl=en i have sheet the cells filled in data pulled via apps script, website. pull in exact same data, not. on old sheets version use able emails when data "changes" different value. new sheets, email when new data pasted cells, though exact same data......and, can't set @ ranges anymore, entire document. there easy way watch cells when data changes, , email myself? this looks lot you're trying do. for checking whether value has changed after edit, this gives necessary information it. hence, know whether new value same 1 or not. implementing "workaround" talked in second link, able compare , send email myself if value changed. initiate onedit trigger goes off onedit function uses these lines after checks in place have passed. var email = "xyz@whatever.com"; mailapp.sendemail(email, "triggermail", "a change has occurred in sheet"); note: sends "notification...
Read more

c++ - Difference between pre and post decrement in recursive function argument -

i have following sample code used pre-decrement void function(int c) { if(c == 0) { return; } else { cout << "dp" << c << endl; function(--c); cout << c << endl; return; } } this function give output input 4 : dp3 dp2 dp1 dp0 0 1 2 3 but when use post decrement void function(int c) { if(c == 0) { return; } else { cout << "dp" << c << endl; function(c--); cout << c << endl; return; } } output goes in infinite loop dp4 can explain me in detail why happens ? this happens because function(c--) called same value of c , when finishes c decremented. recursively called, hence called same value no chance of return , hit stack overflow error. assume this, int x = 1, y = 0; y = x++ result y == 1 , x == 2. if y = ++x , bot x , y 2.
Read more

javascript - IE11 incompatibility with jQuery's 'readonly'? -

the code i'm running has no issues in firefox, or chrome. user typing 1 textbox, tabs next , can continue typing that. when run webpage on ie11, user tabs next textbox , can't type in. textbox has double-clicked before user can insert anything. i've googled , seems ie give problems when using older jquery (i.e. prefers 'prop' 'attr' ). in code below jquery's readonly keyword being used in way ie11 no longer recognise/accept? i assume issue readonly , removeclass('ignore') working expected. //when focus on textbox 1 $("#field1").focus(function() { //remove readonly property textbox 1 $('#field1').prop('readonly', false).removeclass('ignore'); //add readonly property other textboxes $('#field2, #field3, #field4') .prop('readonly', 'readonly').addclass('ignore').val(''); }); i'm using jquery 2.1.3 ...
Read more