Senior Data Engineer (Hybrid: New York, NY - US)

About the position Interested in joining a growing company where you will work with talented colleagues, enhance a supportive and energetic culture, and be part of the climate solution? At Energy Solutions, we focus on the big impacts. And we believe that market-based programs can be a powerful force to deliver large-scale energy, carbon, and water-use savings. Since 1995, we’ve harnessed that power to offer proven, performance-based solutions for our utility, government, and institutional customers. We are currently seeking a Senior Data Engineer to join our Information Systems team to design, develop, and maintain data platforms that support the data needs across Energy Solutions. In this role, the ideal candidate is a passionate and highly skilled professional with expertise in analytics tools and cloud technologies like AWS, Azure or similar technologies. They should be proficient in programming languages such as SQL, NoSQL, and Python, capable of processing large data sets to deliver high-quality, customer-facing data solutions and insights. This unique position is perfect for individuals with technical prowess in data field who want to have an impact on energy efficiency markets and greenhouse gas reductions through our work for major North American utilities and other clients around the country. This is a hybrid work opportunity. ES has offices in Oakland, CA, Orange, CA, Portland, OR, Chicago, IL, and Boston, MA. Responsibilities • Build, automate, and manage near-real-time scalable data ingestion pipelines for master data management, deep-learning, and predictive analytics. • Build and maintain cloud native big data environments on AWS, that are highly secure, scalable, flexible, and highly performant using appropriate SQL, NoSQL and NewSQL technologies. • Lead data governance and data profiling efforts to ensure data quality and proper metadata documentation for data lineage. • Provide technical input into build/buy/partner decisions for all components of the data infrastructure. • Partner closely with Data Scientists, BI developers, and Product Managers to design and implement data models, database schemas, data structures, and processing logic to support various data science, analytics, machine learning, and BI initiatives. • Design and develop ETL (extract-transform-load) processes to validate and transform data, calculate metrics, and model features, populate data models etc., using Spark, Python, SQL, and other technologies in the AWS. • Lead data governance and data profiling efforts to ensure data quality and proper metadata documentation for data lineage. • Lead by example, demonstrating best practices for code development and optimization, unit testing, arenaflex/CD, performance testing, capacity planning, documentation, monitoring, alerting, and incident response to ensure data availability, data quality, and usability. • Define SLAs for data availability and correctness. Automate data availability and quality monitoring and respond to alerts when data delivery SLAs are not being met. • Communicate progress across organizations and levels from individual contributor to executive. Identify and clarify the critical few issues that need action and drive appropriate decisions and actions. Communicate results clearly. Requirements • A bachelor’s degree in computer science or information technology plus 8 years minimum of relevant experience. • High proficiency in programming languages commonly used in ETL development, such as PLSQL, SQL, Python. Ability to write efficient SQL queries, SQL store procedures, develop scripts for data transformations, and utilize programming frameworks and libraries to create/enhance ETL mappings and workflows. • Expertise in utilizing AWS services, including but not limited to arenaflex s3, glue, data catalog, arenaflex redshift, redshift arenaflex and arenaflex arenaflex. Ability to leverage these services to build scalable, reliable, and performant data pipelines and analytics solutions. • Proficiency in working with relational databases such as Postgres, Oracle, MySQL, or SQL Server. Knowledge of database design, optimization techniques, and advanced querying capabilities. • Experience in performance tuning and optimizing database operations. • Familiarity with data governance frameworks and data security best practices. • Passion for learning new technologies, staying up to date with industry trends, and exploring innovative approaches to ETL development. Benefits • Compensation is commensurate with experience and includes a generous retirement package. • Energy Solutions provides an excellent benefits package including medical, dental and vision insurance, other pre-tax contribution plans and an Employee Stock Ownership Plan (ESOP). Apply tot his job

Back to blog

Common Interview Questions And Answers

1. HOW DO YOU PLAN YOUR DAY?

This is what this question poses: When do you focus and start working seriously? What are the hours you work optimally? Are you a night owl? A morning bird? Remote teams can be made up of people working on different shifts and around the world, so you won't necessarily be stuck in the 9-5 schedule if it's not for you...

2. HOW DO YOU USE THE DIFFERENT COMMUNICATION TOOLS IN DIFFERENT SITUATIONS?

When you're working on a remote team, there's no way to chat in the hallway between meetings or catch up on the latest project during an office carpool. Therefore, virtual communication will be absolutely essential to get your work done...

3. WHAT IS "WORKING REMOTE" REALLY FOR YOU?

Many people want to work remotely because of the flexibility it allows. You can work anywhere and at any time of the day...

4. WHAT DO YOU NEED IN YOUR PHYSICAL WORKSPACE TO SUCCEED IN YOUR WORK?

With this question, companies are looking to see what equipment they may need to provide you with and to verify how aware you are of what remote working could mean for you physically and logistically...

5. HOW DO YOU PROCESS INFORMATION?

Several years ago, I was working in a team to plan a big event. My supervisor made us all work as a team before the big day. One of our activities has been to find out how each of us processes information...

6. HOW DO YOU MANAGE THE CALENDAR AND THE PROGRAM? WHICH APPLICATIONS / SYSTEM DO YOU USE?

Or you may receive even more specific questions, such as: What's on your calendar? Do you plan blocks of time to do certain types of work? Do you have an open calendar that everyone can see?...

7. HOW DO YOU ORGANIZE FILES, LINKS, AND TABS ON YOUR COMPUTER?

Just like your schedule, how you track files and other information is very important. After all, everything is digital!...

8. HOW TO PRIORITIZE WORK?

The day I watched Marie Forleo's film separating the important from the urgent, my life changed. Not all remote jobs start fast, but most of them are...

9. HOW DO YOU PREPARE FOR A MEETING AND PREPARE A MEETING? WHAT DO YOU SEE HAPPENING DURING THE MEETING?

Just as communication is essential when working remotely, so is organization. Because you won't have those opportunities in the elevator or a casual conversation in the lunchroom, you should take advantage of the little time you have in a video or phone conference...

10. HOW DO YOU USE TECHNOLOGY ON A DAILY BASIS, IN YOUR WORK AND FOR YOUR PLEASURE?

This is a great question because it shows your comfort level with technology, which is very important for a remote worker because you will be working with technology over time...