Senior Data Engineer
Discover

Crystal Lake, Illinois

Posted in Retail
6 days ago


Job Info


Discover. A brighter future.

With Discover, you'll have the chance to make a difference at one of the world's leading digital banking and payments companies. From Day 1, you'll do meaningful work you're passionate about, with the support and resources you need for success. We value what makes each employee unique and provide a collaborative, team-based culture that gives everyone an opportunity to shine. Be the reason millions of people find a brighter financial future, while building the future you want, here at Discover.

Job Description

As a Senior Data Engineer, you will provide technical (design & development) leadership to the Payments Data Ops Team in development of Extract/Transform/Load (ETL) applications that will interface with all key Discover applications.

Position requires excellent communication skills for understanding business vision and the ability to translate the vision into technical artifacts. Strong technical analysis and design background is also a must-have to ensure that technical deliverables are providing flexible, architecturally sound infrastructure with the ability to reuse in future Data Stores, analytical and operational in AWS.

Responsibilities:

  • Develop data driven solutions utilizing current and next generation technologies to meet evolving business needs.
  • Design, develop, test, and implement data-driven solutions to meet business requirements
  • Ability to quickly identify an opportunity and recommend possible technical solutions.
  • Develop application systems that comply with the standard system development methodology and concepts for design, programming, backup, and recovery to deliver solutions that have superior performance and integrity.
  • Contribute to determining programming approach, tools, and techniques that best meet the business requirements.
  • Understand and follow the PDP process to develop, deploy and deliver the solutions.
  • Be pro-active and diligent in identifying and communicating design and development issues.
  • Provide business analysis and develop ETL code and scripting to meet all technical specifications and business requirements according to the established designs.
  • Offer system support as part of a support rotation with other team members.
  • Utilize multiple development languages/tools such as Python, Spark to build prototypes and evaluate results for effectiveness and feasibility.
  • Operationalize open source data-analytic tools for enterprise use.
  • Develop real-time data ingestion and stream-analytic solutions leveraging technologies such as Kafka, Apache Spark, Python and AWS based solutions.
  • Custom Data pipeline development (Cloud and locally hosted)
  • Work heavily within the Cloud ecosystem and migrate data from Teradata to AWS based platform.
  • Provide support for deployed data applications and analytical models by being a trusted advisor to Data Scientists and other data consumers by identifying data problems and guiding issue resolution with partner Data Engineers and source data providers.
  • Provide subject matter expertise in the analysis, preparation of specifications and plans for the development of data processes.
  • Optimize the performance of ETL processes and scripts by working with other technical staff as needed
  • Ensure proper data governance policies are followed by implementing or validating Data Lineage, Quality checks, classification, etc.
  • Designs and develops data ingestion frameworks, real time processing solutions, and data processing/transformation frameworks leveraging open source tools.
  • Deploys application code and analytical models using CI/CD tools and techniques and provides support for deployed data applications and analytical models.
  • Provides senior level technical consulting to peer data engineers during design and development for highly complex and critical data projects

Minimum Qualifications

At a minimum, here's what we need from you:

  • Bachelors degree in Information Technology, or related field
  • 2+ years of work experience in Data Platform Administration/Engineering, or related

Desired Qualifications

If we had our say, we'd also look for:

  • 4+ years of work experience in Data Platform Administration/Engineering, or related
  • Hands on experience with Amazon Web Services (AWS) based solutions such as Lambda, Dynamo dB, Snowflake and S3.
  • Knowledge of Data Warehouse technology (Unix/Teradata/Ab Initio/Python/Spark/Snowflake/No SQL)
  • Experience in migrating ETL processes (not just data) from relational warehouse Databases to AWS based solutions.
  • Experience in building & utilizing tools and frameworks within the Big Data ecosystem including Kafka, Spark, and NoSQL.
  • Deep knowledge and very strong in SQL and Relational Databases
  • Knowledge of Data Warehouse technology (Unix/Teradata/Ab Initio)
  • Willingness to continuously learn & share learnings with others
  • Ability to work in a fast-paced, rapidly changing environment
  • Very strong verbal & written communication skills
  • Experience within the Financial industry

#LI-KE

What are you waiting for? Apply today!

The same way we treat our employees is how we treat all applicants - with respect. Discover Financial Services is an equal opportunity employer (EEO is the law). We thrive on diversity & inclusion. You will be treated fairly throughout our recruiting process and without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or veteran status in consideration for a career at Discover.



picture of a car

Want to save money on car insurance?


Compare rates instantly

Rates starting at $19/mo

Check my rates *Our Adpolicy

More jobs for you in Retail

Retail

Posted about 2 hours ago

The Fresh Market
Retail

Posted about 2 hours ago

The Fresh Market
Retail

Posted about 2 hours ago

The Fresh Market
View Retail jobs »

Weekly blog post from our employment blog

New Post
3 Tips in Asking for a Job Reference

3 Tips in Asking for a Job Reference

When you apply for a job, typically you are required to have a job reference. Many people ask someone to do a reference for them, but the key ...


Read full blog post
Share this job with the community

Click a community link below, and then social share the Senior Data Engineer job.


African American Job Search Logo
Asian Job Search Logo
Disabled Job Seekers Logo
Hispanic Job Exchange Logo
LGBT Job Search Logo
Seniors to Work Logo
US Diversity Job Search Logo
Veteran Job Center Logo