AI Engineer Expert Auditor, Tech Residency
Coding Temple
RemoteRemote$150 - $200 an hourPart Time
Job Description
*AI Engineer Expert Auditor, Tech Residency (TR)*
*About the Role*
You’ve shipped production AI systems. You’ve debugged hallucinating agents at 2 AM. You’ve stared at eval metrics until the tradeoffs became second nature - and you’ve made the call on what’s ready to deploy and what isn’t.
Now, you want to use that hard-won judgment to set the standard for what “job-ready” means in AI engineering.
At Coding Temple, we’re building the most effective career mobility engine in the world. At the center is *Tech Residency (TR)* - a flagship product that replicates the environment, expectations, and workflows of modern enterprise AI teams.
TR isn’t a course. It’s a *high-stakes, stage-gated verification sprint* - designed to surface real engineering capability under pressure. Residents don’t just “learn.” They build production-grade AI systems from scratch, defend their design decisions, and produce an Employer-Trusted Proof Pack that hiring managers can assess in under 10 minutes.
Your role? *Be the trust layer.*
As AI Engineer Expert Reviewer, you’ll conduct the weekly audits that determine whether each resident’s work meets the bar. You’ll evaluate system architecture, code quality, evaluation rigor, security posture, and operational readiness - scoring against a published rubric and issuing pass/remediate/fail decisions.
Your sign-off is the signal employers trust. If you say it’s verified, it means something.
*What You’ll Do*
*Weekly Audit & Evaluation*
* Conduct formal weekly audits of each resident’s system design, code, evals, and operational readiness against a published rubric
* Issue pass/remediate/fail decisions at each stage gate with clear, actionable feedback
* Verify that all artifacts are reproducible from a clean environment - if it doesn’t reproduce, it doesn’t count
* Review recorded technical defenses and evaluate residents’ ability to explain and defend tradeoffs
*Expert Office Hours & Mentorship*
* Lead weekly 90-minute live office hours (Wednesdays) where residents present challenges and get real-time technical feedback
* Provide async feedback via Slack, PR reviews, and Zoom throughout the week (~3.5 hrs/week)
* Model the thinking patterns and rigor of a senior AI engineer - not by teaching, but by evaluating and questioning
*Rubric & Standards Development*
* Collaborate with the Principal Product Manager to refine evaluation rubrics aligned with the AIUC-1 enterprise trust framework
* Define what “good” vs. “fail” looks like across architecture, data/privacy, security, reliability, safety, and observability
* Ensure rubric categories stay current with evolving enterprise AI standards and hiring manager expectations
*Proof Pack Certification*
* Perform final auditor sign-off on each graduate’s Employer-Trusted Proof Pack
* Write scored assessments including strengths, red flags, and reviewer notes that employers will read
* Ensure every Proof Pack meets the reviewability standard: a hiring manager can assess credibility in ≤10 minutes
*Credibility & Employer Trust*
* Represent the technical credibility of the TR product - your name and judgment are what employers trust
* Participate in Demo Day panels and employer-facing events as needed
* Provide signal to the team on market trends, hiring manager expectations, and evolving AI engineering standards
*Who You Are*
*Experience*
* 6+ years of professional engineering experience (not research-only)
* 2+ years building LLM/AI systems in production (2023+ era)
* Have shipped at least one production AI system with: RAG + evals, tool-calling/agent workflows, monitoring + cost/latency controls, and failure mode handling
* Comfortable reviewing PRs, tearing apart system designs, and writing clear “ready/not ready” assessments
* Experience designing evaluation harnesses and test plans for AI systems
* Bonus: experience in workforce development, technical interviewing, or building assessment rubrics
*Skills*
* Deep expertise in system design & tradeoffs (cost/latency/accuracy)
* Strong understanding of AI-specific security: prompt injection defenses, tool-call authorization, PII handling, secrets management
* Proficiency in reliability engineering: fallbacks, evals, monitoring, drift detection
* Can evaluate code quality, evidence quality, and communication quality with equal rigor
* Strong written communicator - your audit notes will be read by hiring managers
*Mindset*
* You believe “trust” is earned through evidence, not credentials
* You’re rigorous but fair—you hold a high bar and help people clear it
* You’ve been on-call for your own systems and know what production accountability feels like
* You’re motivated by raising the standard for what “job-ready” means in AI
*Success in This Role Looks Like*
* Every Proof Pack you certify is trusted by hiring managers—zero credibility gaps
* Residents receive clear, actionable feedback that accelerates their growth week over week
* Stage-gate pass/fail decisions are consistent, defensible, and aligned with the published rubric
* Employer review-to-interview conversion rates improve as Proof Pack quality increases
* TR is recognized as the gold standard in employer-trusted AI verification
*Why Join Us*
* Define what “job-ready AI engineer” actually means - aand build the system that proves it
* Work with a team that values trust, autonomy, and high standards
* Use your production experience to change lives - every audit you run helps someone land their next career
* Flexible, part-time engagement (~15 hrs/week) with deep purpose and visible impact
*Compensation*
* Engagement: Contract / Part-Time (~15 hours/week per cohort)
* Rate: $150–$250/hour, commensurate with experience and production credibility
* Commitment: 8-week cohort cycles with opportunity for ongoing engagement across multiple cohorts
* Equity: Eligible for advisor-level equity consideration for ongoing engagements
If interested email: *[email protected]*
*About Coding Temple*
Coding Temple exists to unlock potential.
We help people do the most meaningful work of their lives, and help companies build the teams that make that possible.
We prepare individuals with the skills, confidence, and real-world experience to succeed from day one. For employers, we deliver job-ready talent trained to meet the demands of modern work. Whether it’s a career change or a hiring challenge, we exist to close the gap between potential and opportunity - because when that happens, lives change and companies grow.
*Our Vision*
We envision a world where anyone, regardless of age, origin, or socioeconomic status, can achieve financial independence and professional fulfillment.
*Our Mission*
Our mission is to *Empower Prosperity* by delivering capabilities to professionals and organizations that drive economic growth.
We do this by creating a *Virtuous Organization*, a place where employees derive deep meaning from their work and gain increasing passion and energy from their colleagues, customers, students, and collective results. Together, we create an environment where each person can do their very best work.
Our core values guide us:
* Gratitude & Appreciation
* Dignity & Respect
* Support & Compassion
* Caring & Concern
* Meaningfulness & Purpose
* Inspiration & Positive Energy
* Forgiveness & Understanding
* Trust & Integrity
Job Types: Full-time, Part-time
Pay: $150.00 - $200.00 per hour
Expected hours: 15 – 45 per week
Work Location: Remote
*About the Role*
You’ve shipped production AI systems. You’ve debugged hallucinating agents at 2 AM. You’ve stared at eval metrics until the tradeoffs became second nature - and you’ve made the call on what’s ready to deploy and what isn’t.
Now, you want to use that hard-won judgment to set the standard for what “job-ready” means in AI engineering.
At Coding Temple, we’re building the most effective career mobility engine in the world. At the center is *Tech Residency (TR)* - a flagship product that replicates the environment, expectations, and workflows of modern enterprise AI teams.
TR isn’t a course. It’s a *high-stakes, stage-gated verification sprint* - designed to surface real engineering capability under pressure. Residents don’t just “learn.” They build production-grade AI systems from scratch, defend their design decisions, and produce an Employer-Trusted Proof Pack that hiring managers can assess in under 10 minutes.
Your role? *Be the trust layer.*
As AI Engineer Expert Reviewer, you’ll conduct the weekly audits that determine whether each resident’s work meets the bar. You’ll evaluate system architecture, code quality, evaluation rigor, security posture, and operational readiness - scoring against a published rubric and issuing pass/remediate/fail decisions.
Your sign-off is the signal employers trust. If you say it’s verified, it means something.
*What You’ll Do*
*Weekly Audit & Evaluation*
* Conduct formal weekly audits of each resident’s system design, code, evals, and operational readiness against a published rubric
* Issue pass/remediate/fail decisions at each stage gate with clear, actionable feedback
* Verify that all artifacts are reproducible from a clean environment - if it doesn’t reproduce, it doesn’t count
* Review recorded technical defenses and evaluate residents’ ability to explain and defend tradeoffs
*Expert Office Hours & Mentorship*
* Lead weekly 90-minute live office hours (Wednesdays) where residents present challenges and get real-time technical feedback
* Provide async feedback via Slack, PR reviews, and Zoom throughout the week (~3.5 hrs/week)
* Model the thinking patterns and rigor of a senior AI engineer - not by teaching, but by evaluating and questioning
*Rubric & Standards Development*
* Collaborate with the Principal Product Manager to refine evaluation rubrics aligned with the AIUC-1 enterprise trust framework
* Define what “good” vs. “fail” looks like across architecture, data/privacy, security, reliability, safety, and observability
* Ensure rubric categories stay current with evolving enterprise AI standards and hiring manager expectations
*Proof Pack Certification*
* Perform final auditor sign-off on each graduate’s Employer-Trusted Proof Pack
* Write scored assessments including strengths, red flags, and reviewer notes that employers will read
* Ensure every Proof Pack meets the reviewability standard: a hiring manager can assess credibility in ≤10 minutes
*Credibility & Employer Trust*
* Represent the technical credibility of the TR product - your name and judgment are what employers trust
* Participate in Demo Day panels and employer-facing events as needed
* Provide signal to the team on market trends, hiring manager expectations, and evolving AI engineering standards
*Who You Are*
*Experience*
* 6+ years of professional engineering experience (not research-only)
* 2+ years building LLM/AI systems in production (2023+ era)
* Have shipped at least one production AI system with: RAG + evals, tool-calling/agent workflows, monitoring + cost/latency controls, and failure mode handling
* Comfortable reviewing PRs, tearing apart system designs, and writing clear “ready/not ready” assessments
* Experience designing evaluation harnesses and test plans for AI systems
* Bonus: experience in workforce development, technical interviewing, or building assessment rubrics
*Skills*
* Deep expertise in system design & tradeoffs (cost/latency/accuracy)
* Strong understanding of AI-specific security: prompt injection defenses, tool-call authorization, PII handling, secrets management
* Proficiency in reliability engineering: fallbacks, evals, monitoring, drift detection
* Can evaluate code quality, evidence quality, and communication quality with equal rigor
* Strong written communicator - your audit notes will be read by hiring managers
*Mindset*
* You believe “trust” is earned through evidence, not credentials
* You’re rigorous but fair—you hold a high bar and help people clear it
* You’ve been on-call for your own systems and know what production accountability feels like
* You’re motivated by raising the standard for what “job-ready” means in AI
*Success in This Role Looks Like*
* Every Proof Pack you certify is trusted by hiring managers—zero credibility gaps
* Residents receive clear, actionable feedback that accelerates their growth week over week
* Stage-gate pass/fail decisions are consistent, defensible, and aligned with the published rubric
* Employer review-to-interview conversion rates improve as Proof Pack quality increases
* TR is recognized as the gold standard in employer-trusted AI verification
*Why Join Us*
* Define what “job-ready AI engineer” actually means - aand build the system that proves it
* Work with a team that values trust, autonomy, and high standards
* Use your production experience to change lives - every audit you run helps someone land their next career
* Flexible, part-time engagement (~15 hrs/week) with deep purpose and visible impact
*Compensation*
* Engagement: Contract / Part-Time (~15 hours/week per cohort)
* Rate: $150–$250/hour, commensurate with experience and production credibility
* Commitment: 8-week cohort cycles with opportunity for ongoing engagement across multiple cohorts
* Equity: Eligible for advisor-level equity consideration for ongoing engagements
If interested email: *[email protected]*
*About Coding Temple*
Coding Temple exists to unlock potential.
We help people do the most meaningful work of their lives, and help companies build the teams that make that possible.
We prepare individuals with the skills, confidence, and real-world experience to succeed from day one. For employers, we deliver job-ready talent trained to meet the demands of modern work. Whether it’s a career change or a hiring challenge, we exist to close the gap between potential and opportunity - because when that happens, lives change and companies grow.
*Our Vision*
We envision a world where anyone, regardless of age, origin, or socioeconomic status, can achieve financial independence and professional fulfillment.
*Our Mission*
Our mission is to *Empower Prosperity* by delivering capabilities to professionals and organizations that drive economic growth.
We do this by creating a *Virtuous Organization*, a place where employees derive deep meaning from their work and gain increasing passion and energy from their colleagues, customers, students, and collective results. Together, we create an environment where each person can do their very best work.
Our core values guide us:
* Gratitude & Appreciation
* Dignity & Respect
* Support & Compassion
* Caring & Concern
* Meaningfulness & Purpose
* Inspiration & Positive Energy
* Forgiveness & Understanding
* Trust & Integrity
Job Types: Full-time, Part-time
Pay: $150.00 - $200.00 per hour
Expected hours: 15 – 45 per week
Work Location: Remote