GitOps: Infrastructure as Code with Git for Microservices

Discover GitOps principles and practices for managing microservices infrastructure using Git as the single source of truth.

published: reading time: 25 min read author: GeekWorkBench

GitOps: Infrastructure as Code with Git for Microservices

Managing dozens or hundreds of microservices without a coherent strategy gets messy fast. You need to know what is deployed where, reproduce environments reliably, and recover from failures without digging through manual steps. GitOps solves this by applying version control practices to infrastructure.

The approach took off in the Kubernetes world, but the principles work beyond any single platform. If you run microservices at scale, GitOps changes how you think about deployments, reliability, and team workflows.

Introduction

GitOps takes DevOps best practices, version control, collaboration, and compliance, and applies them to infrastructure automation. The core idea: use Git as the single source of truth for application code and infrastructure configuration.

When your entire system state lives in Git, every infrastructure change goes through a pull request. You get peer review, history, and rollback with a single command. Deployment becomes auditable by default, not as an afterthought.

Weaveworks coined the term, but the principles spread across the industry. Kubernetes shops found GitOps compelling because it matches how Kubernetes works.

Traditional infrastructure management often relies on scripts or manual processes. Run a command here, change a config there, and gradually production drifts from what anyone intended. GitOps flips this. Instead of pushing from CI, you define desired state and a controller makes reality match.

Core Concepts

GitOps rests on three principles that set it apart from other infrastructure approaches.

Declarative Configuration

Everything in your infrastructure is expressed declaratively. Rather than scripts that run steps, you define what the system should look like. Kubernetes manifests, Helm charts, and infrastructure-as-code templates fit this model. Your Git repository becomes a complete specification of how systems should run.

Versioned and Immutable

Every change produces a new version in Git history. You never modify existing commits; changes go in as new commits. This immutability gives you rollback and audit. If something breaks, you trace what changed, who approved it, and revert to a known-good state.

Pull-Based Reconciliation

A GitOps operator watches the state in Git and compares it to what runs in your cluster. When drift happens, the operator pulls changes to fix it. This contrasts with traditional CI/CD pipelines, where deployment tools push to clusters from outside.

GitOps Operators: ArgoCD and Flux

Two tools dominate GitOps for Kubernetes: ArgoCD and Flux. Both use pull-based reconciliation but differ in style.

ArgoCD, now a CNCF graduated project, gives you a GUI alongside its reconciliation engine. It shows application state, diffs between desired and actual, and offers rollback through the UI. Teams like ArgoCD for visibility across multiple clusters.

Flux takes a more Git-native approach, designed to fit developer workflows. It uses Custom Resources to define deployments. Flux v2 added modularity with progressive delivery and multi-tenancy.

The choice comes down to team preference. ArgoCD suits teams wanting operational visibility. Flux suits teams prioritizing Git-native workflows and programmatic control. Either works fine.

Repository Structure

How you organize Git repos affects how well GitOps works. Two patterns dominate.

Monorepo Approach

Some teams put everything in one repository. Application manifests, infrastructure configs, environment overrides all together. This simplifies management and makes cross-cutting changes easy. But it creates contention when multiple teams work there, and access control gets coarse.

App Repo and Environment Repo Pattern

A more common approach separates concerns. Each microservice has its own repo with code and Kubernetes manifests. A separate environment repo holds configuration for each environment, references app repos, and defines how services compose.


app-repo/
  ├── deployment.yaml
  ├── service.yaml
  └── ingress.yaml

environment-repo/
  ├── production/
  │   ├── namespace.yaml
  │   ├── configmap.yaml
  │   └── kustomization.yaml
  └── staging/
      ├── namespace.yaml
      ├── configmap.yaml
      └── kustomization.yaml

This separation creates boundaries. Teams iterate on services independently while platform teams manage environments centrally.

Kustomize and Helm both work well in GitOps workflows. Kustomize does lightweight patching. Helm handles sophisticated templating and packaging. Pick based on your complexity needs.

Secret Management with GitOps

Secrets trip up most GitOps implementations. GitOps wants everything in Git, but you cannot commit plaintext secrets. Several approaches handle this.

Sealed Secrets from Bitnami

Sealed Secrets encrypts using public-key cryptography. You encrypt a Secret manifest locally with the Sealed Secrets controller public key, then commit the encrypted version. The controller in your cluster decrypts and creates the actual Secret. Encrypted secrets live in Git, but only your cluster reads them.

HashiCorp Vault Integration

Vault gives you centralized secret management with dynamic credentials, access control, and audit logging. GitOps operators can fetch secrets at deployment time. This keeps secrets out of Git but adds a Vault dependency.

External Secrets Operator

The External Secrets Operator bridges Kubernetes Secrets with external secret stores. It defines an ExternalSecret resource pointing to your secret store. The operator syncs values into the cluster as standard Kubernetes Secrets. Works with AWS Secrets Manager, GCP Secret Manager, Azure Key Vault.

Each approach trades convenience against security and complexity. Sealed Secrets is simple but needs key management. Vault is powerful but adds infrastructure. External Secrets is flexible but introduces another component to run.

Drift Detection and Automatic Reconciliation

Configuration drift happens. Someone runs kubectl directly. A helm upgrade changes values unintentionally. A node failure reschedules pods with different configs. GitOps operators watch for this drift and fix it.

The operator periodically fetches desired state from Git and compares it to actual cluster state. When differences appear, it applies the desired configuration to restore alignment.

This automated correction helps. Manual mistakes get fixed automatically. Consistent state becomes the default. You stop waking up to snowflakes that drifted overnight.

ArgoCD can also watch external resources beyond Kubernetes, tracking Terraform or CloudFormation alongside application manifests.

GitOps Flow Diagram

The following diagram shows the typical GitOps workflow and how changes move through the system.


graph TD
    A[Developer] -->|Push Code| B[Application Repo]
    A -->|Pull Request| C[Environment Repo]
    B -->|Image Build| D[Container Registry]
    C -->|ArgoCD/Flux| E[GitOps Operator]
    D -->|Image Tag Update| C
    E -->|Sync| F[Kubernetes Cluster]
    F -->|State Check| E
    G[Secrets Vault] -->|Fetch Secrets| E

Developers push code to application repos or submit pull requests to environment repos. When code pushes, CI builds container images and updates image tags in the environment repo. The GitOps operator detects the change, validates against cluster state, and syncs the desired configuration.

Comparison with Traditional CI/CD Push Model

Traditional CI/CD uses a push model. CI builds an image, then uses kubectl or helm to push directly to clusters. Credentials live in CI systems. Deployments come from external systems connecting inward.

GitOps flips this. A GitOps operator inside your cluster pulls changes and applies them. Credentials do not leave the cluster for deployments. The cluster controls when to sync, not an external system.

The push model works fine for simpler setups. GitOps shines when managing multiple clusters, prioritizing security, or needing continuous reconciliation. Pick based on your actual needs, not hype.

Benefits of GitOps

GitOps changes how teams manage infrastructure.

Strong Audit Trail

Every change flows through Git pull requests. You get complete history of who changed what, when, why. Code review gates infrastructure changes. Compliance that demands change tracking is satisfied naturally.

Quick Rollback

Reverting a bad change means reverting a Git commit and waiting for the operator to sync. This works even if the bad change deployed problematic code. Rollback time drops from minutes of manual work to seconds of Git operations.

Improved Developer Experience

Developers work with infrastructure through familiar Git workflows. No need to learn kubectl or remember deployment procedures. Reviewing infrastructure changes feels like reviewing code.

Consistency Across Environments

The same manifests can target multiple environments. Promoting to production means updating a reference, not manually re-applying configs. Environment parity improves.

Security

Clusters do not need inbound access from deployment tools. The operator pulls changes, removing an attack vector. Credentials stay in the cluster instead of circulating to external systems.

Common Pitfalls / Anti-Patterns

GitOps adds complexity you have to handle.

Secret Management

Secrets need extra tooling. Committing secrets to Git is not an option. Each secret strategy adds operational overhead.

Large Monorepo Performance

When everything lives in one repo, operations slow down. Git history grows large, diffs become unwieldy, merge conflicts multiply. Careful repo design helps, but it is an ongoing problem.

Learning Curve

Teams used to imperative tools need time to think declaratively. Understanding reconciliation and debugging when it fails takes effort.

Operator Maintenance

The GitOps operator becomes critical infrastructure. Keeping it updated, monitoring its health, and troubleshooting issues needs attention.

When to Use / When Not to Use

GitOps solves specific problems but adds complexity that only pays off in certain contexts.

When to Use GitOps

Use GitOps when:

  • Running Kubernetes in production with multiple clusters
  • Compliance requires complete audit trails for infrastructure changes
  • Teams need self-service deployments without cluster credentials
  • Environment consistency matters across development, staging, and production
  • Rollback speed is critical for incident response
  • Multiple teams deploy to shared clusters
  • You want single source of truth for both application and infrastructure state

Use GitOps for multi-cluster management when:

  • Managing multiple environments (dev, staging, production) with similar configurations
  • Running across multiple cloud providers or regions
  • Disaster recovery requires rapid environment reproduction
  • On-call engineers need quick visibility into cluster state

When Not to Use GitOps

Consider alternatives when:

  • Single cluster, single application, simple deployment needs
  • Team is small and all members have direct cluster access
  • Existing CI/CD pipeline already works well for deployments
  • Strictly imperative infrastructure management is required (rare but valid cases)
  • Learning curve would block adoption before benefits materialize
  • External systems manage cluster state directly (some managed Kubernetes offerings)

GitOps vs Alternatives Trade-offs

ApproachBest ForLimitations
GitOps (ArgoCD/Flux)Declarative infra, audit trails, multi-clusterLearning curve, additional operators to maintain
CI/CD Push ModelSimple setups, existing CI toolingCredentials outside cluster, no drift correction
Infrastructure as Code (Terraform)Cloud provisioning, heterogeneous infraState management, not Kubernetes-native
kubectl scriptsOne-off operations, emergenciesNo audit trail, not repeatable, manual
Helm-onlyApplication templating without GitOpsNo automatic sync, no UI, limited visibility

Tool Selection: ArgoCD vs Flux


graph TD
    A[GitOps Operator Selection] --> B{Team Priorities}
    B -->|Operational Visibility| C[ArgoCD]
    B -->|Git-Native Workflows| D[Flux]
    B -->|Multi-Tenant Setup| E[ArgoCD]
    B -->|Minimal Footprint| F[Flux]
    C --> G[GUI, Application CRD, Rollback UI]
    D --> H[Controller CRDs, CLI, Programmatic]
    E --> G
    F --> H

Production Failure Scenarios

GitOps failures can leave clusters in inconsistent states or block deployments entirely. Understanding these scenarios helps you design resilient GitOps workflows.

Common GitOps Failures

FailureImpactMitigation
GitOps operator crashCluster drifts without correctionHigh availability deployment, monitoring
Repository branch mismatchWrong version deployed to clusterBranch protection, PR reviews, environment gates
Image tag confusionUnknown version running, hard to rollbackUse git SHA tags, immutable image references
Secret encryption key rotationSealed secrets become unreadableAutomated key rotation with backup keys
Helm chart dependency failureDeployment fails, partial stateLockfile for dependencies, test upgrades
Cluster connectivity lossOperator cannot sync, drift accumulatesLocal cache for offline operation, alerts
Ingress/traffic split mismatchApp unreachable after deployHealth checks, canary analysis, traffic monitoring
CRD version mismatchCustom resources fail to applyTest in staging first, version pinning

Drift Detection Failures


graph TD
    A[GitOps Reconciliation Loop] --> B[Fetch Desired State]
    A --> C[Compare to Cluster State]
    B --> D{Diff Found?}
    D -->|Yes| E[Apply Diff]
    D -->|No| F[Healthy]
    E --> G[Apply Succeeded?]
    G -->|No| H[Mark Out of Sync]
    G -->|Yes| I[Sync Successful]
    H --> J[Alert Team]
    I --> A
    J --> K[Manual Intervention]

Secret Management Failures

FailureImpactMitigation
Sealed Secrets private key lostCannot decrypt secrets, services failBackup keys in secure location, key rotation
Vault unavailable during deployPods fail to start, deployment blocksCache secrets locally, graceful degradation
External Secrets sync failureStale or missing secretsRetry logic, alerting on sync errors
Secret rotation during deployApplication crashes with old secretGraceful secret reload, zero-downtime rotation

Recovery Procedures


# Force sync ArgoCD application
argocd app sync user-service --force

# Rollback via ArgoCD
argocd app rollback user-service

# Check Flux reconciliation status
flux reconcile kustomization user-service --with-source

# Suspend Flux reconciliation (for emergency)
flux suspend kustomization user-service

# View GitOps operator logs
kubectl logs -n argocd deploy/argocd-server

# Check sealed secrets controller
kubectl get pods -n kube-system -l name=sealed-secrets

# Manual secret decryption (emergency)
kubectl exec -n kube-system -l name=sealed-secrets -- unseal

Multi-Cluster GitOps Failures

FailureImpactMitigation
Cluster-specific config driftOne cluster differs from othersCluster fleet auditing, compliance checks
Cross-cluster dependency mismatchService A in cluster 1 incompatible with Service B in cluster 2Contract testing, version synchronization
Network partition between clustersGitOps operator isolated, drift accumulatesRegional Git repos, local caching

Quick Recap

Key Takeaways

  • Pull-based reconciliation keeps clusters matching desired state automatically
  • ArgoCD suits teams wanting operational visibility; Flux suits Git-native workflows
  • Secret management is the hardest part: Sealed Secrets, Vault, or External Secrets all have trade-offs
  • Repository structure matters: app repos + environment repos pattern scales well
  • Drift detection and automatic correction are GitOps superpowers

Implementation Readiness Assessment


# Verify ArgoCD CLI access
argocd cluster list

# Check application sync status
argocd app list

# Validate Kubernetes manifests in repo
kubectl apply --dry-run=server -f k8s/

# Test Helm template rendering
helm template myapp ./charts/myapp --debug

# Verify sealed secrets controller
kubectl get crd sealedsecrets.bitnami.com

# Check Flux CRDs installed
kubectl get crd | grep flux

# Validate Git repo structure
git ls-tree -r HEAD --name-only | head -50

Pre-Adoption Checklist

  • Kubernetes clusters provisioned and accessible
  • Git repository structure designed (app repos, environment repos, or monorepo)
  • Secret management strategy selected and implemented
  • CI pipeline configured to build and push container images
  • GitOps operator (ArgoCD or Flux) installed and configured
  • Initial application manifests committed and deploying
  • Rollback procedure documented and tested
  • Monitoring for GitOps operator health configured
  • Team training completed on GitOps workflows

GitOps Drift and Troubleshooting

A critical production scenario: an engineer runs kubectl edit deployment directly to hotfix a crashing pod. The GitOps operator detects the drift and reconciles — reverting the manual change back to the Git-desired state. The pod crashes again. This cycle repeats until someone notices. The root cause: the desired state in Git was wrong, but the manual fix was never committed.

Mitigation: Implement drift detection alerts that notify the team when manual changes are reverted. Use ArgoCD’s CompareOptions to ignore specific fields that may be managed externally. Most importantly, enforce a policy: every production change must go through Git first.

Trade-offs: Push-Based vs Pull-Based GitOps

AspectPush-Based (CI/CD)Pull-Based (ArgoCD/Flux)
SecurityCredentials live in CI; wider attack surfaceCredentials stay in cluster; minimal exposure
ComplexitySimpler to set up; familiar CI patternsRequires operator deployment and configuration
ToolingWorks with any CI system (GitHub Actions, Jenkins)Requires ArgoCD, Flux, or similar operator
Drift detectionNone — CI pushes once and forgetsContinuous — operator detects and corrects drift
Network modelOutbound from CI to clusterInbound pull from cluster to Git
Multi-clusterMust configure each cluster in CISingle Git repo can drive many clusters
RollbackManual or separate pipelineInstant — revert Git commit

Security and Compliance Considerations

  • Never commit plaintext secrets to Git. Use Sealed Secrets, SOPS + age, External Secrets Operator, or Vault integration.
  • RBAC for Git-to-cluster sync: The GitOps operator should run with the minimum Kubernetes RBAC permissions needed. Use namespace-scoped service accounts rather than cluster-admin.
  • Audit trail: Every sync event should be logged with the git commit SHA, the operator identity, and the resources changed. ArgoCD provides this natively; Flux requires external logging.
  • Branch protection: Protect the Git branch that drives production. Require PR reviews, status checks, and signed commits. A compromised Git repo equals a compromised cluster.
  • Secret rotation: When secrets rotate in Vault or AWS Secrets Manager, the External Secrets Operator should reconcile automatically. Test rotation procedures regularly — stale secrets are a common cause of GitOps deployment failures.
  • Compliance note: For SOC 2 and ISO 27001, the Git commit history serves as your change management audit trail. Ensure commit messages link to change tickets and that PR approvals are documented.

Interview Questions

1. What is the core principle of GitOps, and how does it differ from traditional CI/CD deployment approaches?

Expected answer points:

  • GitOps uses Git as the single source of truth for declarative infrastructure desired state
  • Pull-based reconciliation: operator inside the cluster pulls changes, rather than CI/CD pushing from external systems
  • Traditional CI/CD uses push model where credentials live in CI systems and deployments originate from external tools
  • GitOps enables continuous reconciliation — the operator continuously ensures actual state matches desired state in Git
2. Explain the three foundational principles of GitOps and how each contributes to infrastructure reliability.

Expected answer points:

  • Declarative Configuration: All infrastructure expressed as desired state (Kubernetes manifests, Helm charts), not imperative scripts
  • Versioned and Immutable: Every change produces a new Git commit; existing commits never modified — enables rollback and full audit trail
  • Pull-Based Reconciliation: GitOps operator continuously compares desired state (in Git) against actual cluster state, auto-correcting drift
3. Compare ArgoCD and Flux as GitOps operators. When would you choose one over the other?

Expected answer points:

  • ArgoCD: CNCF graduated, provides GUI for operational visibility, shows application state and diffs, suitable for teams needing cross-cluster visibility and rollback UI
  • Flux: Git-native design using Custom Resources, fits developer workflows, modular with progressive delivery support, better for Git-first teams preferring CLI and programmatic control
  • Choose ArgoCD when: multi-cluster management, operational dashboards needed, teams want UI visibility
  • Choose Flux when: Git-native workflows, CLI preference, minimal footprint, teams comfortable with Custom Resources
4. Describe the app-repo and environment-repo pattern. What advantages does this separation provide?

Expected answer points:

  • App repo: Each microservice has its own repository containing code and Kubernetes manifests (deployment.yaml, service.yaml, ingress.yaml)
  • Environment repo: Holds configuration per environment (production/staging), references app repos, defines how services compose together
  • Benefits: Team autonomy — services iterate independently; environment consistency — platform teams manage environments centrally; clear boundaries reduce contention
5. What are the main approaches for handling secrets in GitOps, and what are their trade-offs?

Expected answer points:

  • Sealed Secrets: Encrypts Secret manifests using public-key cryptography; controller in cluster decrypts; encrypted secrets live in Git; simple but requires key management
  • HashiCorp Vault: Centralized secret management with dynamic credentials and audit logging; operator fetches secrets at deploy time; powerful but adds infrastructure dependency
  • External Secrets Operator: Bridges Kubernetes Secrets with external stores (AWS Secrets Manager, GCP Secret Manager, Azure Key Vault); syncs values as standard K8s Secrets; flexible but introduces another component
6. How does GitOps handle configuration drift, and why is automatic reconciliation important?

Expected answer points:

  • Drift occurs when: someone runs kubectl directly, Helm upgrades change values unintentionally, node failures reschedule pods with different configs
  • GitOps operator periodically fetches desired state from Git, compares to actual cluster state, applies corrections when differences found
  • Automatic reconciliation: Manual mistakes get fixed automatically, consistent state becomes default, teams avoid waking up to snowflakes that drifted overnight
7. What are the security benefits of pull-based GitOps over push-based CI/CD?

Expected answer points:

  • No inbound access required: Clusters do not need inbound access from deployment tools; operator pulls changes from within
  • Credentials stay in cluster: Deployment credentials do not leave the cluster or circulate to external CI systems
  • Reduced attack surface: External CI systems with credentials are eliminated as deployment vectors
  • Audit trail: Every change goes through Git PRs with documented approvals
8. Explain the GitOps rollback process. How does it compare to traditional deployment rollback approaches?

Expected answer points:

  • GitOps rollback: Revert a Git commit, operator detects change and syncs previous desired state — works even if bad code was deployed
  • Traditional rollback: Minutes of manual work (re-running deploy scripts, coordinating with teams, potential for human error)
  • GitOps rollback time: Seconds of Git operations; ArgoCD `app rollback` or Flux `reconcile` with `--with-source`
  • Consistency: Rollback uses same mechanism as deployment, ensuring repeatability
9. What challenges should teams anticipate when adopting GitOps, and how can they mitigate them?

Expected answer points:

  • Secret management complexity: Requires additional tooling (Sealed Secrets, Vault, or External Secrets Operator); never commit plaintext secrets
  • Large monorepo performance: Git history grows large, diffs unwieldy, merge conflicts multiply; careful repo design essential
  • Learning curve: Teams accustomed to imperative tools need time to think declaratively; reconciliation debugging requires new skills
  • Operator maintenance: GitOps operator becomes critical infrastructure; must monitor health, keep updated, troubleshoot issues
10. When would you recommend NOT using GitOps? What alternatives should teams consider?

Expected answer points:

  • Single cluster, single application with simple deployment needs — GitOps overhead may not justify benefits
  • Small teams with direct cluster access — existing CI/CD may be sufficient
  • Strictly imperative infrastructure requirements — valid in rare cases where declarative model does not fit
  • Managed Kubernetes offerings that manage cluster state externally
  • Alternatives: CI/CD push model (simpler setups), Terraform for cloud provisioning (heterogeneous infra), kubectl scripts (one-off operations), Helm-only (application templating without automatic sync)
11. How does GitOps integrate with CI/CD pipelines? Describe the relationship between image builds and GitOps synchronization.

Expected answer points:

  • CI/CD handles build and test — compiles code, runs tests, builds container images, pushes to registry
  • GitOps handles deployment — when CI updates image tag in Git manifests, GitOps operator detects change and syncs to cluster
  • Typical flow: Developer pushes code → CI builds image → CI updates image tag in environment repo → GitOps operator pulls changes and deploys
  • Separation of concerns: CI focuses on "is the code good?" while GitOps focuses on "is the right version deployed?"
12. Explain how GitOps handles multi-cluster deployments. What patterns exist for managing dozens of clusters with a single operator?

Expected answer points:

  • Application of Practice pattern: One set of manifests in Git drives multiple clusters; operator in each cluster syncs the same desired state
  • Cluster fleet management: ArgoCD's ApplicationSet controller or Flux's multi-tenancy features scale to many clusters
  • Environment promotion: code → staging → production via Git branch or kustomization overlays
  • Compliance made easy: All clusters can be guaranteed to run identical configurations or known variants
13. Describe disaster recovery scenarios specific to GitOps. How does GitOps help teams recover from infrastructure failures?

Expected answer points:

  • Git commit history is the backup: Any previous infrastructure state can be recreated by checking out an older commit
  • New cluster recovery: Point GitOps operator at existing repo, and the entire desired state gets recreated automatically
  • Fast RTO (Recovery Time Objective): Rather than recreating infrastructure manually, git revert triggers full reconciliation
  • Repository corruption risk: Keep Git repo redundant across multiple remotes; lost state is recreatable from known-good commits
14. How does GitOps change team workflows and responsibilities? What new roles or skills does it require?

Expected answer points:

  • Platform engineers own the GitOps operator and environment repos; developers own app repos and submit PRs for changes
  • Self-service deployments: Developers do not need cluster credentials — PR to environment repo triggers deployment
  • Shift from imperative to declarative thinking: Teams express desired state rather than step-by-step procedures
  • New skills needed: Git workflows, PR review processes, reading Kubernetes YAML, understanding reconciliation
15. What monitoring and alerting should teams set up for GitOps operators? How do you know when reconciliation fails?

Expected answer points:

  • Operator health: Pod status, CPU/memory metrics, restart count for ArgoCD or Flux controllers
  • Sync status: Alerts when application is out of sync (ArgoCD provides SyncStatus metrics; Flux provides Kustomization conditions)
  • Drift alerts: Notify when operator corrects unexpected manual changes — indicates policy violation
  • Commit-to-sync latency: Alert if desired state change does not reflect in cluster within expected window
16. Compare rollback strategies in ArgoCD versus Flux. How does each tool handle version history and reverting deployments?

Expected answer points:

  • ArgoCD rollback: `argocd app rollback` reverts to previous sync revision; stores history of sync operations; GUI shows history and diffs
  • Flux rollback: `flux reconcile kustomization --with-source` re-reads from Git; revert Git commit for previous desired state
  • GitOps rollback is deterministic — reverts desired state in Git, operator syncs to match
  • Both support `--force` flag for emergency situations where desired state must override cluster reality
17. What Git branching strategies work best with GitOps? How do teams manage environment promotion through branches or overlays?

Expected answer points:

  • Branch-per-environment: `main` → `staging` → `production` branches; risky because promotion requires merge conflicts
  • Overlay pattern (recommended): Single source of truth in `main`, Kustomize overlays for `staging/` and `production/` directories
  • Tag-based promotion: Git tags mark release candidates; operators watch specific refs
  • Pull request workflow: Every environment change goes through PR review; automated CI validates before merge
18. How does GitOps handle partial or failed deployments? What happens when a Helm chart has dependency issues mid-deploy?

Expected answer points:

  • GitOps operators track resource health — Helm install that partially succeeds shows as "out of sync" or "degraded"
  • ArgoCD: Sync waves and hooks control deployment order; health checks verify before proceeding
  • Flux: Kustomization dependencies and ready conditions gate progression
  • Failure isolation: Use `FailFast` in ArgoCD sync policy to stop on first error; or allow partial sync with `Prune=false` to avoid data loss
19. What are the scaling challenges for GitOps at enterprise level? How do organizations manage thousands of applications across hundreds of clusters?

Expected answer points:

  • ArgoCD ApplicationSet controller: Generates applications from templates, scales to thousands of app definitions via generator CRs
  • Flux multi-tenancy: Namespace-scoped operators with GitRepository and Kustomization per tenant; avoids cluster-wide permissions
  • Monorepo performance: Large repos cause slow git clones; consider repo per team or sparse checkout strategies
  • Operator resource limits: Each operator needs CPU/memory headroom when reconciling many resources simultaneously
20. Design a GitOps migration strategy for a team currently using imperative kubectl scripts and manual Helm deployments.

Expected answer points:

  • Phase 1 — Inventory: Document all existing deployments, Helm releases, configmaps, and secrets that need to be captured
  • Phase 2 — Repo structure: Set up app repos and environment repo pattern; commit existing manifests as-is (no changes yet)
  • Phase 3 — Pilot operator: Install ArgoCD or Flux in staging; point at new repos; validate reconciliation matches current state
  • Phase 4 — Incremental cutover: Migrate one service at a time; disable old CI deploy steps; enable GitOps sync
  • Phase 5 — Full adoption: Remove direct cluster access for developers; enforce all changes through Git PRs; document rollback procedures

Further Reading

Cross-Roadmap References

Conclusion

GitOps changes how teams manage microservices infrastructure. By using Git as the source of truth with pull-based reconciliation, you get auditability, fast rollback, and environment consistency. The approach fits Kubernetes well since both use declarative models.

Adopting GitOps requires upfront investment in tooling, repo structure, and team training. For teams running multiple clusters or wanting operational visibility, it pays off. Even for smaller deployments, the discipline GitOps forces around version control and review helps.

If you run Kubernetes and have not tried GitOps, ArgoCD or Flux are worth exploring. Start small with one application, see how reconciliation works in practice, then expand.

Category

Related Posts

GitOps: Declarative Deployments with ArgoCD and Flux

Implement GitOps for declarative, auditable infrastructure and application deployments using ArgoCD or Flux as your deployment operator.

#gitops #argocd #flux

Kustomize: Native Kubernetes Configuration Management

Use Kustomize for declarative Kubernetes configuration management without Helm's templating—overlays, patches, and environment-specific customization.

#kubernetes #kustomize #devops

Health Checks: Liveness, Readiness, and Service Availability

Master health check implementation for microservices including liveness probes, readiness probes, and graceful degradation patterns.

#microservices #health-checks #kubernetes