scala - Changing an attribute in an object that belongs to RDD -
i have following code :
def generatestoriesnew(outputpath: string, groupedrdd:rdd[(string,iterable[string])], isinchurnmode: boolean, isinchurnperiod: boolean) { val windowedrdd = groupedrdd.map(some code) var windowedrdd2 = windowedrdd.filter(r => r != null).map(a=>a.churnperiod(isinchurnperiod,isinchurnmode)) val prettystringrdd = windowedrdd2.map(r => { r.tostring }) prettystringrdd.saveastextfile(outputpath)
}
and here code churnpriod function:
def churnperiod( churnperiod:boolean, churnmode: boolean): unit = { if (churnmode && rooteventtype.equalsignorecase("c")){ var churncuststory: custstoryn = null var nonchurncuststory: custstoryn = null var churnperiodeventstory: mutable.mutablelist[storyeventn] = null var nonchurneventstory: mutable.mutablelist[storyeventn] = null churnperiodeventstory = new mutable.mutablelist[storyeventn] nonchurneventstory = new mutable.mutablelist[storyeventn] var lasteventchurnperiod = true var currenteventstory = eventstory var max = currenteventstory.length println(max); if (currenteventstory.size > 0) { (i <- 0 until currenteventstory.length) { var currentevent = currenteventstory(i) if (currentevent.timesenserootevent < 90) { churnperiodeventstory.+=(currentevent) //lasteventchurnperiod = true } else { nonchurneventstory.+=(currentevent) lasteventchurnperiod = false } } } if (churnperiod) eventstory = churnperiodeventstory else eventstory=null
} }
but churn period function not change eventstory member of custstory class. missing here ?
class custstoryn (val custid:string, var rooteventtype:string, var rooteventtime:long, var eventstory:mutable.mutablelist[storyeventn])
my hypothesis either: 1.map not right transformation function have 2.churnperiod function never called 3.i can not change eventstory member of cust story class have idea how can troubleshoot problem?
this trivial determine debugging. put few breakpoints , can see if program stops inside function, , transformations occur.
my guess problem line: if (currenteventstory.size > 0)
, thus, list starts @ size 0 remains @ size 0 forever. option churnperiod
never true, thus, compute lot never assign eventstory
variable.
your code need cleanup ;-)
Comments
Post a Comment