Key Responsibilities :

• Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities
  
• Implementing ETL process 
  
• Monitoring performance and advising any necessary infrastructure changes
  
• Defining data retention policies
  

Skills and Qualifications :

• Proficient understanding of distributed computing principles
  
• Management of Hadoop cluster, with all included services 
  
• Ability to solve any ongoing issues with operating the cluster
  
• Proficiency with Hadoop v2, MapReduce, HDFS
  
• Experience with building stream-processing systems, using solutions such as Storm or Spark-Streaming
  
• Good knowledge of Big Data querying tools, such as Pig, Hive, and Impala
  
• Experience with Spark
  
• Experience with integration of data from multiple data sources
  
• Experience with NoSQL databases, such as HBase, Cassandra, MongoDB is a plus
  
• Knowledge of various ETL techniques and frameworks, such as Flume
  
• Experience with various messaging systems, such as Kafka or RabbitMQ
  
• Good understanding of Lambda Architecture, along with its advantages and drawbacks
  
• Experience with Cloudera/MapR/Hortonworks is a plus
  
• Bachelor’s or Master’s degree in computer science or software engineering
  

Additional qualifications:

• To have excellent oral and written communication skills
  
• To be able to work in teams and collaborate with others to clarify requirements
  
• To have strong co-ordination and project management skills to handle complex projects.
  
• To be able to assist in documenting requirements as well as resolve conflicts or ambiguities
  
• To enjoy being challenged and to solve complex problems on a daily basis