amazon web services - AWS Lambda subscribed to SNS topic doesn't process all messages to the topic -

i building alerting system powered amazon services.

i drop file onto s3 daily spawns lambda function (let's call generator function) process file.

generator builds alerts based on file , posts multiple messages sns topic (let's call outbox) - 1 message each recipient calculated generator.

i have second lambda function (let's call courier) subscribed outbox should take each message , it.

the generator code:

// 'generator' function  exports.handler = function (event, context) {   console.log('reading options event:\n', util.inspect(event, {depth: 5}));    var users = {};   var usersubscriptions = {};   var alerts = {};   var artists = {};   var tracks = {};    async.waterfall([     function downloadsubscribersfile (next) {       // stuff     },     function downloadandformatactionsfile (next) {       // download data file , analyse     },     function publishalerts (next) {       // have alerts built, need mail them out       var recipients = object.keys(alerts);        async.each(recipients, function (recipient, callback) {         var recipientalert = alerts[recipient];         console.log(util.inspect(recipientalert, { depth: 10 }));          if (alerts[recipient].actions.artists.length < 1) {           return callback();         }          var params = {           topicarn: sns_topic_arn,           subject: recipient,           message: json.stringify(recipientalert)         };          sns.publish(params, function (err, data) {           if (err) {             console.log(err);             return callback(err);           } else {             console.log('published message: \n', util.inspect(data, { depth: 10 }));             console.log('message was: \n', util.inspect(params, { depth: 10 }));           }            return callback();         })       }, function (err) {         if (err) return next(err);         next();        })     }   ], function (err) {     if (err) {       console.log('error: ', err);     } else {       console.log('process successful');     }     context.done();   })  } 

and other function:

// 'courier' function  console.log('loading function');  exports.handler = function(event, context) {     console.log(json.stringify(event, null, 2));     console.log('from sns:', event.records[0].sns.message);     context.succeed(); }; 

when generator function invoked, can see 12 messages should posted sns topic. there no errors recorded when publishing these messages, , yet courier function fires once.

i'm wondering if anyone's had similar issues , whether there's i'm missing here. there's haven't configured correctly in aws i'm pretty confident set should be.

---

update:

after looking @ messages i'm attempting send, seems message picked sns seemed 1 smallest payload. i'm wondering if sns able cope lots of small, frequent messages topic...?

yes, sns can cope high throughput on single topic. however, there maximum message size 256kb, if messages larger might cause.

i see generator function logging messages, seeing 12 logged messages message ids? see have variable alerts hoping multiple recipients from, don't see setting it.

my advice: add more logging before sent message verify think should happening happening.