General Knowledge, Skills & Abilities
- Be a good detail-oriented data engineer
- Systematic and organizational skills important.
- Willing to commit for completing deliverable on time.
- Must have experience with Spark, Hive, Scala or py spark.
- Preferred experience in one of the following technologies: Nifi, Kafka, or any other streaming technologies.
- 3+ years experience in data engineering building ETL pipelines using JAVA or Python or Scala
- Should be good at Pig, HIVE scripting.
- Solid understanding of HDFS is important.
- Work experience within a Data Warehousing/Business Intelligence/Data analytics group, and have hand’s-on experience with Hadoop
- Create tables/views in Hive or other relevant scripting language
- Have experience with Agile development methodologies
- Experience with NoSQL databases, such as HBase, Cassandra, MongoDB.
Experience Architecting Solutions Utilizing any of the following:
- JAVA or Python or Scala programming languages
- Nifi, Kafka-topics, or any other streaming technologies
- Parquet/Avro/ORC/XML/JSON/ORC/CSV/TXT formats