ELT Developer
Cloud and Things, Inc
Albany, NY 12205$140,000 - $155,000 a year
Job Description
Our goal is to solve problems and deliver results for our clients. At Cloud and Things, you can be a part of transforming the public sector’s IT environment. Our team is on the forefront of helping to solve the government's most complex IT challenges. If you are seeking a role that offers the opportunity to work on rewarding projects, consider a career with Cloud and Things.??
*This is an exempt position. Salary commensurate with experience*
Overview:
Location: Albany, NY - Hybrid
Salary: $140,000 - $155,000
We are seeking an ELT Developer who will support our NYS client with the design and development of Service based data integration architectures. This role will be expected to build and maintain data pipelines and a centralized integration hub, using Databricks (PySpark and Databricks SQL) to handle batch, incremental, CDC, and event-driven data loads. The ideal candidate is a senior, hands-on developer and technical lead with strong SQL/API integration experience, and a track record of ensuring secure, well-governed, high-performing data integrations.
Duties:
Design and Develop Service based data integration architectures.
Build ELT pipelines with Databricks using PySpark and Databricks SQL
Implement batch and incremental data loads, including CDC patterns.
Create reusable, standardized integration patterns.
Establish logical and physical integration layers, curation, and distribution.
Work with cross-functional teams (developers, product managers, etc.) to define integration requirements and ensure successful integration.
Identify and resolve integration related issues, optimize performance, and ensure data security.
Create and maintain clear and comprehensive documentation for internal users and stakeholders.
Mandatory Qualifications:
7+ years of experience working on complex projects with 2 or more years in a leadership role as a Developer
7+ years of experience in designing end-to-end architecture for an enterprise Data Integration hub for centralized data ingestion and transformation for various ingestion patterns (batch and event-driven)
7+ years proven experience with SQL/NoSQL databases, API development (REST/SOAP), version control (Git), and building data integrations between on-premises & cloud systems
7+ years of proven experience with standardizing API consumption, error handling, re-tires, and throttling.
7+ years of experience in managing schema evolution and backward compatibility supporting legacy transformations
7+ years of hands-on experience in designing and implementing data integration hub
7+ years of proven experience in complex API based workflows, data governance, metadata and lineage tooling
7+ years of experience in integrating Informatica with Databricks as the hubs’ transformation and curation engine
7+ years of experience in designing hub-based data publishing mechanisms for data warehouses and analytics
7+ years proven experience with security standards (OAuth, SAML), data governance, performance tuning, and relevant cloud/integration certifications
7+ years of experience in event streaming platforms and cloud integration architectures
Associates degree or combination of experience and education
Cloud and Things complies with all applicable federal, state, and local laws regarding recruitment and hiring. All qualified applicants are considered for employment without regard to race,?color, religion, sex, sexual orientation, gender identity, national origin, age, disability, protected veteran status, or any other category protected by applicable federal, state, or local laws.?
AI-Assisted Resume Evaluation Notice
Cloud and Things – Talent Management
Notice to Candidates
Cloud and Things utilizes artificial intelligence (AI) tools to assist our recruiting team in evaluating candidate applications for streamlining; consistency, efficiency, and thoroughness. All hiring decisions are ultimately made by our human recruiting professionals.
How AI Is Used
Our AI tools assist by:
Analyzing resumes against job requirements
Supporting our recruiters in candidate data evaluation
Ensuring consistent review standards across all applications
Important: AI serves as a support tool only. As noted above, all candidate selection and hiring decisions are made by experienced human recruiters. Your unedited resume will be processed by our AI tools as part of this evaluation.
Your Data and Privacy
Cloud and Things Data Handling:
Your information is processed securely and used exclusively for recruitment purposes
Cloud and Things may store your resume in our Applicant Tracking System (ATS) indefinitely for future job matching opportunities
You may opt out of long-term ATS storage by emailing your name and your request to opt out of storing your resume in the ATS to: [email protected]
All personal information is handled confidentially in accordance with our privacy policy
AI Tool Data Processing:
AI processing data is retained for a maximum of 90 days, after which it is deleted
All data sent to AI tools is encrypted in transit and at rest
AI tools comply with applicable privacy laws including GDPR and CCPA
Personal data is anonymized or minimized wherever possible during AI processing
Your Participation
By submitting your application, you acknowledge this notice and consent to AI-assisted evaluation as part of our recruitment process. You may opt out only by choosing not to submit your resume for consideration.
*This is an exempt position. Salary commensurate with experience*
Overview:
Location: Albany, NY - Hybrid
Salary: $140,000 - $155,000
We are seeking an ELT Developer who will support our NYS client with the design and development of Service based data integration architectures. This role will be expected to build and maintain data pipelines and a centralized integration hub, using Databricks (PySpark and Databricks SQL) to handle batch, incremental, CDC, and event-driven data loads. The ideal candidate is a senior, hands-on developer and technical lead with strong SQL/API integration experience, and a track record of ensuring secure, well-governed, high-performing data integrations.
Duties:
Design and Develop Service based data integration architectures.
Build ELT pipelines with Databricks using PySpark and Databricks SQL
Implement batch and incremental data loads, including CDC patterns.
Create reusable, standardized integration patterns.
Establish logical and physical integration layers, curation, and distribution.
Work with cross-functional teams (developers, product managers, etc.) to define integration requirements and ensure successful integration.
Identify and resolve integration related issues, optimize performance, and ensure data security.
Create and maintain clear and comprehensive documentation for internal users and stakeholders.
Mandatory Qualifications:
7+ years of experience working on complex projects with 2 or more years in a leadership role as a Developer
7+ years of experience in designing end-to-end architecture for an enterprise Data Integration hub for centralized data ingestion and transformation for various ingestion patterns (batch and event-driven)
7+ years proven experience with SQL/NoSQL databases, API development (REST/SOAP), version control (Git), and building data integrations between on-premises & cloud systems
7+ years of proven experience with standardizing API consumption, error handling, re-tires, and throttling.
7+ years of experience in managing schema evolution and backward compatibility supporting legacy transformations
7+ years of hands-on experience in designing and implementing data integration hub
7+ years of proven experience in complex API based workflows, data governance, metadata and lineage tooling
7+ years of experience in integrating Informatica with Databricks as the hubs’ transformation and curation engine
7+ years of experience in designing hub-based data publishing mechanisms for data warehouses and analytics
7+ years proven experience with security standards (OAuth, SAML), data governance, performance tuning, and relevant cloud/integration certifications
7+ years of experience in event streaming platforms and cloud integration architectures
Associates degree or combination of experience and education
Cloud and Things complies with all applicable federal, state, and local laws regarding recruitment and hiring. All qualified applicants are considered for employment without regard to race,?color, religion, sex, sexual orientation, gender identity, national origin, age, disability, protected veteran status, or any other category protected by applicable federal, state, or local laws.?
AI-Assisted Resume Evaluation Notice
Cloud and Things – Talent Management
Notice to Candidates
Cloud and Things utilizes artificial intelligence (AI) tools to assist our recruiting team in evaluating candidate applications for streamlining; consistency, efficiency, and thoroughness. All hiring decisions are ultimately made by our human recruiting professionals.
How AI Is Used
Our AI tools assist by:
Analyzing resumes against job requirements
Supporting our recruiters in candidate data evaluation
Ensuring consistent review standards across all applications
Important: AI serves as a support tool only. As noted above, all candidate selection and hiring decisions are made by experienced human recruiters. Your unedited resume will be processed by our AI tools as part of this evaluation.
Your Data and Privacy
Cloud and Things Data Handling:
Your information is processed securely and used exclusively for recruitment purposes
Cloud and Things may store your resume in our Applicant Tracking System (ATS) indefinitely for future job matching opportunities
You may opt out of long-term ATS storage by emailing your name and your request to opt out of storing your resume in the ATS to: [email protected]
All personal information is handled confidentially in accordance with our privacy policy
AI Tool Data Processing:
AI processing data is retained for a maximum of 90 days, after which it is deleted
All data sent to AI tools is encrypted in transit and at rest
AI tools comply with applicable privacy laws including GDPR and CCPA
Personal data is anonymized or minimized wherever possible during AI processing
Your Participation
By submitting your application, you acknowledge this notice and consent to AI-assisted evaluation as part of our recruitment process. You may opt out only by choosing not to submit your resume for consideration.