数据工程师
Data Engineer
Requirements
Bachelor's degree in Computer Science or a related field.
At least 7 years of IT experience of which at least 5 years' experience in similar role, ideally in a Big Data tools
Proficient understanding of distributed computing principles
Strong experience of Hadoop, Spark, Iceberg and Spring Boot
Experience with building stream-processing systems, using solutions such as Spark-Streaming
Good knowledge of Big Data querying tools, such as Pig, Hive, and Impala
Experience with integration of data from multiple data sources
Experience with NoSQL databases, such as HBase, Cassandra, MongoDB
Knowledge of various ETL techniques and frameworks
Experience with various messaging systems, such as Kafka or RabbitMQ
Knowlege with Big Data ML toolkits, such as Mahout, SparkML, or H2O
Good understanding of Lambda Architecture, along with its advantages and drawbacks
Originally posted on Himalayas