<story><title>Show HN: s3-lambda – Lambda functions over S3 objects: each, map, reduce, filter</title><url>https://github.com/littlstar/s3-lambda</url></story><parent_chain></parent_chain><comment><author>hoodoof</author><text>Its weird how S3 seems to be the unwanted stepchild of AWS.&lt;p&gt;So many obvious innovations just aren&amp;#x27;t turning up.&lt;p&gt;For example, strangely, AWS introduced tagging for S3 resources, but you can&amp;#x27;t search&amp;#x2F;filter by tag, nor is the tag even returned when you get a list of objects, you can only get the tag with an object request. The word &amp;quot;pointless&amp;quot; springs to mind.&lt;p&gt;In fact it&amp;#x27;s strange that there is NO useful filtering at all apart from the very useful folder&amp;#x2F;hierarchy&amp;#x2F;prefix filtering.  But apart from that you can&amp;#x27;t do wildcard searches or filters or date filters or tag filters.&lt;p&gt;I&amp;#x27;m building an application right now that needs to get a list of all the jpg files - the only way to do that is get every single object in the bucket and manually filter out the unwanted ones - feels like its 1988 again.&lt;p&gt;It seems like it would also be valuable for there to be alternate interfaces to S3 such as the ability to send data via ftp or SMTP or sftp or whatever, but there are no such interfaces.&lt;p&gt;Hopefully Google will goad AWS into action on S3 innovation by implementing such features.</text></comment>
<story><title>Show HN: s3-lambda – Lambda functions over S3 objects: each, map, reduce, filter</title><url>https://github.com/littlstar/s3-lambda</url></story><parent_chain></parent_chain><comment><author>simonw</author><text>First impression: this is a  brilliant piece of software design.&lt;p&gt;The ability to compose a map&amp;#x2F;filter chain and execute it in parallel against every object in an S3 bucket that matches a specific prefix - wow.&lt;p&gt;The set of problems that can be quickly and cheaply solved with this thing is enormous. My biggest problem with lambda functions is that they are a bit of a pain to actually write - for transforming data in S3 this looks like my ideal abstraction.</text></comment>