Fast Streaming Solution

High level view:

Web API          -> Kafka                     -> Storm (Streaming)

                    -> Hadoop/ HDFS    -> MR/ Hive (Batch)

 

Specifics TBD..

Lftp

Lftp a handy utility for all kinds of ftp, sftp, and other file transfer use-cases from the command-line on a *nix system. Give it a shot if there's ever a need..

Atlasian Stack

 - Jira
- Confluence
Among many others.. All well integrated enhance dev productivity...

Teradata

Teradata busy getting a chunk of the BigData pie. Teradata Parrallel Transporter (TPT) and Adv. SQL Combo makes querying Big Data sources fast and efficient using state of the art caching and other optimizations.