Design, implement, automate our Big Data Platform for real-time and batch processing.
Automate the deployment, upgrade and monitoring operations on the platform.
Identify and experiment with new tools and technologies to optimize the efficiency, scalability and reliability of the platform.
Desired skills and background 1
Proven experience in designing, building and optimizing a Big data platform with Spark, Storm, or equivalent technologies.
Experience with TB-level daily ingestion and real-time processing, PB-level data analytics is a big plus.
A solid understanding of the big data landscape, the evolution of associated technologies along with their advantages and disadvantages.
Strong programming skills in Java, Scala, and / or Python.
Experience with one of the large cloud-computing infrastructure solutions like AWS, Openstack, Azure etc.
Able to benchmark systems, analyze system bottlenecks and make proposals to resolve issues
Big Data Architect