Data Engineer General
Posted on: June 21, 2022
Transforming the Future with the Convergence of Simulation and
Data Data Engineer GeneralJob Summary:Our client in Dearborn, MI is
looking for a Data Engineer General.This is a W-2, contract
position.What You Will Do:
- Data Engineer Design and build production data engineering
solutions to deliver data pipeline patterns using following Google
Cloud Platform (GCP) services:
- In-depth understanding of Google's product technology and
- BigQuery - Warehouse/ data marts - Through understanding of Big
Query internals to write efficient queries for ELT needs, creation
of views/materialized views, creation of reusable store procedures
- DataFlow (Apache Beam) - reusable Flex templates/ data
processing frameworks using Java for both batch and stream
- Pub/Sub, Kafka, Confluent Kafka - Real time streaming of
database changes or events.
- Experience of designing, building, and deploying
production-level data pipelines using Kafka.
- Strong experience working on Event Driven Architecture.
- Strong knowledge of the Kafka Connect framework, with
experience using several connector types HTTP REST proxy, JMS,
File, SFTP, JDBC etc.
- Experience in handling huge volumes of streaming messages from
- Cloud Composer (Apache Airflow) - to build, monitor and
orchestrating the pipeline.
- Knowledge on BigTable.
- Cloud SQL, Compute Engine, Cloud Function, Cloud Run and App
Engine, Cloud Storage.
- Experience with open-source distributed storage and processing
utilities in the Apache Hadoop family.
- Extensive knowledge on processing various file formats orc,
Avro, csv, json, xml etc.
- Knowledge/experience in any ETL tools like
DataStage/Informatica - Ability to understand existing on-premises
ETL workflows and redesign them in GCP.
- Experience and expertise on Terraform to deploy the GCP's in
- Knowledge/ Experience on connecting to on-prem API's from
- Big Query Terraform
- Cloud run
- MUST be willing to work as a W-2How You Will Be Successful:
- Envision the Future
- Communicate Honestly and Broadly
- Seek Technology and Business "First"
- Embrace Diversity and Take RisksWhat We Offer:
- Competitive Salary
- Comprehensive Benefit Package
- 401(k) with matching contributions
- Paid Time Off
- Employee Discounts
- Free training on all Altair productsWhy Work with Us:Altair is
a global technology company providing software and cloud solutions
in the areas of data analytics, product development, and
high-performance computing (HPC). Altair enables organizations in
nearly every industry to compete more effectively in a connected
world, while creating a more sustainable future. With more than
3,000 engineers, scientists, and creative thinkers in 25 countries,
we help solve our customer's toughest challenges and deliver
unparalleled service, helping the innovators innovate, drive better
decisions, and turn today's problems into tomorrow's
opportunities.Our vision is to transform customer decision making
with data analytics, simulation, and high-performance computing.For
more than 30 years, we have been helping our customers integrate
electronics and controls with mechanical design to expand product
value, develop AI, simulation and data-driven digital twins to
drive better decisions, and deliver advanced HPC and cloud
solutions to support unlimited idea exploration. To learn more,
please visit altair.comReady to go? #ONLYFORWARD
Keywords: Altair, Dearborn , Data Engineer General, Engineering , Dearborn, Michigan
Didn't find what you're looking for? Search again!