Big data Engineer Atlanta, GA Any Visa (Except H1B, OPT & CPT)

Hy

I hope you are doing well.

 

I am hiring a consultant for one of my client's requirements. The Job Description for the same is written below. If you find yourself comfortable with the requirement please reply back with your updated resume

 

 

 Position:  Big Data Engineer

 

Location:   Atlanta, GA

 

Duration: 6+ Months

 

Any Visa (Except H1B, OPT & CPT)

 

Work is on-site as needed (up to 3 days per week)

 

The candidate will participate in all phases of the Data Engineering life cycle and will independently and collaboratively write project requirements, architect solutions and perform data ingestion development and support duties.  They must be a very good communicator, both written and verbal, and have some experience working with business areas to translate their business data needs and data questions into project requirements. 

Required:           

• 6+ years of overall IT experience
• 3+ years of experience with high-velocity high-volume stream processing: Apache Kafka and Spark Streaming

• Experience with real-time data processing and streaming techniques using Spark structured streaming and Kafka
• Deep knowledge of troubleshooting and tuning Spark applications

• 3+ years of experience with data ingestion from Message Queues (Tibco, IBM, etc.) and different file formats across different platforms like JSON, XML, CSV
• 3+ years of experience with Big Data tools/technologies like Hadoop, Spark, Spark SQL, Kafka, Sqoop, Hive, S3, HDFS, or
• 3+ years of experience building, testing, and optimizing 'Big Data' data ingestion pipelines, architectures, and data sets
• 2+ years of experience with Python (and/or Scala) and PySpark/Scala-Spark
• 3+ years of experience with Cloud platforms e.g. AWS, GCP, etc.
• 3+ years of experience with database solutions like Kudu/Impala, or Delta Lake or Snowflake or BigQuery
• 2+ years of experience with NoSQL databases, including HBASE and/or Cassandra
• Experience in successfully building and deploying a new data platform on Azure/ AWS
• Experience in Azure / AWS Serverless technologies, like, S3, Kinesis/MSK, lambda, and Glue
• Strong knowledge of Messaging Platforms like Kafka, Amazon MSK & TIBCO EMS or IBM MQ Series
• Experience with Databricks UI, Managing Databricks Notebooks, Delta Lake with Python, Delta Lake with Spark SQL, Delta Live Tables, Unity Catalog 
• Knowledge of Unix/Linux platform and shell scripting is a must
• Strong analytical and problem-solving skills

Preferred:

• Strong SQL skills with ability to write intermediate complexity queries
• Strong understanding of Relational & Dimensional modeling  
• Experience with GIT code versioning software
• Experience with REST API and Web Services
• Good business analyst and requirements gathering/writing skills

 

Kind Regards,

 

Gaurav Pandey| Absolute IT | Recruitment Manager

116 • Village Blvd • Suite 200 • Princeton • New Jersey • 08540

AbsoluteIT Direct: (609) 934 3460 Office: 201-228-3009 EXT 141 Email- gaurav@absoluting.com