The Financial Question—Are We Living Through Another Bubble?

The Investment Surge

Billions of dollars are flowing into AI start-ups, infrastructure, and model providers. As of October 2025, venture capitalists have poured £153 billion ($192.7 billion) into AI start-ups so far this year—setting new global records and putting 2025 on track to be the first year where more than half of total VC dollars went into the industry.[1] In Q3 2025 alone, global venture funding jumped 38% to reach $97 billion, with the three largest rounds going to foundation model companies Anthropic ($13 billion), xAI ($5.3 billion), and Mistral AI ($2 billion).[2]

Established players like Microsoft, Google, and Amazon have committed staggering sums into AI-specific cloud infrastructure and GPU capacity. Microsoft is investing $80 billion in fiscal 2025, Amazon plans $100 billion, and Google is targeting $75 billion—with combined infrastructure investments from tech mega-caps projected to reach $364 billion.[3]

This surge feels eerily reminiscent of the dot-com bubble of 2000—a period when any company adding ".com" to its name could attract outsized valuations. The parallel is clear: FOMO-driven investment, speculative valuations, and promises of a "revolution" that few can fully articulate.

Yet there are differences. Unlike early web start-ups that could spin up overnight, AI at scale demands real physical capital: data centres, chips, fibre networks, and vast operational energy. That infrastructure is tangible, expensive, and (at least somewhat) defensible.

A Rational Bubble?

Could both things be true—that AI is overhyped, and it will transform industries? Possibly. Market bubbles often overstate short-term potential whilst underestimating long-term impact. The internet boom left behind massive overcapacity and bankruptcies, but also the foundations of today's online economy.

A similar pattern might play out here. Some AI start-ups will fade, but the infrastructure and capabilities they helped build could prove essential later.

I'm not qualified to call a definitive financial verdict—but we might be in a "mini-bubble" within a broader structural shift.

The Technical Question—Will AI Deliver?

"It's Just Pattern Matching"

Critics argue that modern AI isn't truly intelligent; it merely recombines statistical patterns from its training data. In this view, large language models (LLMs) are glorified autocomplete systems—powerful but fundamentally shallow. Recent research from Apple and IBM has highlighted these limitations. IBM's analysis notes that LLMs "can't do deductive reasoning" and are instead "set up to do pattern recognition", whilst Apple research demonstrates that AI models primarily rely on statistical pattern matching rather than genuine mathematical reasoning, with accuracy collapsing on high-complexity tasks. Columbia University Professor Vishal Misra has articulated similar concerns, arguing in his June 2025 essay that language models are fundamentally limited by their reliance on statistical patterns and cannot truly improve themselves through reasoning.[4]

The critique is fair: AI systems hallucinate, lack grounding in physical reality, and often fail at causal reasoning. But to dismiss them entirely misses something essential.

Humans Are Pattern Matchers Too

Human cognition relies heavily on pattern recognition. From reading facial expressions to completing sentences, much of our "thinking" involves comparing current stimuli with stored examples. We may encode richer context, but the process is still probabilistic and experience-based.

In that sense, AI's "pattern matching" is not alien—it's familiar. The question isn't whether AI uses patterns, but how far pattern-based learning can go before it hits cognitive or conceptual ceilings.

Counterarguments to "AI is Just Pattern Matching"

There are several compelling counterarguments to this reductive claim, and experts increasingly emphasise that, whilst pattern matching is a fundamental mechanism, modern AI systems operate in ways that go beyond this basic principle:

Hierarchical Reasoning and Abstraction: Advanced AI models leverage hierarchical representations that move beyond raw pattern recognition, enabling abstraction, analogical reasoning, and context-aware responses—facilitating problem-solving that resembles human cognition in specific domains. Recent research on hierarchical reasoning models demonstrates that AI systems inspired by the brain's multi-timescale processing can achieve exceptional performance on complex reasoning tasks, including the Abstraction and Reasoning Corpus (ARC), a key benchmark for measuring artificial general intelligence capabilities.[8]

Generalisation and Zero-Shot Learning: State-of-the-art AI models, especially large language models, demonstrate the ability to generalise from previously unseen data ("zero-shot" tasks), inferring solutions through context rather than simple memorisation of past patterns. Research has shown that large language models are capable of zero-shot reasoning on complex multi-step tasks, performing competently on problems they have never been explicitly trained to solve.[9]

Autonomous Planning and Multi-Step Reasoning: Agentic AI workflows not only identify patterns but plan workflows, adapt to novel data, and synthesise information across unstructured contexts, mimicking some aspects of human decision-making and creative synthesis.

Emergent Behaviours: Modern deep learning systems exhibit emergent behaviours (such as generating code, composing music, or playing strategic games) that were not explicitly trained into them, suggesting a more complex form of intelligence than rote statistical pattern matching. As language models scale up, they demonstrate emergent abilities—capabilities that appear unpredictably at certain model sizes and were not present in smaller versions.[10]

Human Cognition Relies on Patterns Too: Critiques of AI as "just pattern matching" sometimes overlook the reality that human cognition itself relies heavily on identifying, storing, and manipulating patterns; intelligence emerges from using those patterns adaptively.

Whilst pattern matching is a fundamental building block, contemporary AI systems utilise this capacity as a substrate for more sophisticated reasoning, planning, and abstraction—undermining claims that "AI is just pattern matching" and supporting a more nuanced view of how artificial intelligence operates in practice.

Technical Perspective: What Does AI Really Deliver?

In domains rich in structured data and textual context—summarisation, translation, search, code generation, documentation—AI has already proved transformative. Developers routinely use tools such as Amazon Q, Replit, Lovable or Copilot-like assistants to draft, test, and debug code. Analysts use AI to summarise long documents or generate reports in minutes.

These are not trivial gains; they redefine productivity in knowledge work.

Many critics argue that today's AI systems are not "intelligent" in the way humans are, and are merely sophisticated pattern matchers or glorified text generators. The distinction between human and artificial intelligence is subtle: whilst humans certainly use pattern matching to solve problems, what separates us is our awareness of our limitations and our ability to self-correct.

That said, it's undeniable that AI has become remarkably effective in text generation, summarisation, and other language-heavy tasks. This has driven progress in fields like software development automation, customer service response, fraud detection, and medical research—domains where large volumes of data or text need to be managed rapidly and intelligently.

Agentic AI—A New Name for Workflow Automation?

A new term is circulating: agentic AI. It describes systems that can plan, act, and self-correct through multiple steps, rather than executing a single prompt.

On closer inspection, it's reminiscent of workflow automation tools like IFTTT, Zapier, or n8n—systems built around conditional logic: If This Then That. The difference is that agentic AI can dynamically reason about tasks, make decisions under uncertainty, and learn from context.

Traditional automation tools—think of systems like IFTTT or Zapier—operate within rigid, rule-based frameworks. They execute predefined sequences: if X happens, do Y. These systems are dependable but inflexible. If something unexpected occurs—say, a missing input, an API change, or a new context—they typically fail silently or require manual intervention. Their strength lies in predictability, but their weakness is brittleness. They can't reason about why a step failed or decide on an alternative course of action.

More sophisticated platforms like n8n occupy a middle ground. n8n allows for branching, looping, and dynamic adaptation, offering greater flexibility and customisation, particularly for developers building complex automations.[14] Unlike simpler tools, n8n provides a visual interface where different apps connect like puzzle pieces through nodes, and it supports custom code alongside pre-built integrations. However, even n8n operates primarily through structured, event-driven logic—when something happens, something else follows.

Agentic AI represents a step beyond this paradigm. Rather than relying on static rules or even complex conditional logic, agentic systems learn from context. They can interpret goals described in natural language, decompose them into sub-tasks, and plan multi-step actions autonomously. When something goes wrong, they don't just stop—they use reasoning and feedback loops to attempt recovery, such as re-running a step, reformulating a query, or selecting a different tool. This adaptability is often powered by large language models, which allow these agents to parse flexible instructions and dynamically adjust their behaviour. Whilst traditional automation tools like n8n excel at structured workflows and event-driven logic, agentic AI systems are better suited for building intelligent systems where agents can think through problems and assess outcomes.[15]

In short, agentic AI blends the predictability of automation with the fluid intelligence of contextual reasoning—moving automation from a rigid "workflow engine" to something more akin to a digital collaborator.

Where Is the World-Changing AI?

Take Sora 2, OpenAI's text-to-video model. Its results are visually stunning—cinematic clips rendered from short text prompts. Yet beyond creative or entertainment niches, its world-changing impact remains speculative.

Or look at Tesla's Optimus robot. Tesla released footage of Optimus performing kung fu–style movements, demonstrating fluid motion and balance.[11] It's an impressive technical feat, but one designed for controlled conditions. Whether such dexterity translates into robust, generalisable robotics is uncertain.

These are exciting glimpses, not revolutions. The path from demo to deployment is long and fraught with engineering, safety, and economic barriers. But, again, AI innovation is moving at terrific speed.

Whilst AI brings hype and headlines, genuine world-changing applications remain rare. Sora 2, a high-performance multimodal generative model, is technically impressive but hasn't fundamentally transformed daily life. Meanwhile, demonstrations like Tesla's Optimus robot performing kung fu moves—praised for their smooth, humanlike execution—highlight technical progress in machine learning and robotics. However, such feats are often achieved in highly controlled environments and are not yet generalised solutions for complex real-world problems.

Despite major advances across industries—from agriculture to autonomous vehicles and fraud detection—these accomplishments have yet to redefine society on the scale promised by AI's loudest advocates.

The Infrastructure Backdrop—Power, Scale, and Sustainability

The Cost of Intelligence

AI doesn't exist in a vacuum—it runs on silicon and electricity. According to the International Energy Agency, data centres currently account for around 1–1.5% of global electricity consumption.[5] The IEA projects that data centre electricity consumption will more than double by 2030, with AI being a chief driver of this growth.[6] However, data centre demand growth is expected to account for less than 10% of global electricity demand growth between 2024 and 2030.[7] Training a single frontier model can cost tens of millions of pounds and emit thousands of tonnes of CO₂ equivalent.

New hyperscale data centres are under construction worldwide, often near renewable-energy hubs. But the scale raises hard questions:

• How sustainable is exponential growth in compute demand?

• Can innovation in model efficiency outpace energy consumption?

• What happens when power supply becomes a geopolitical bottleneck?

Scaling Limits

Model performance improvements have so far followed predictable scaling laws—but the cost curve is steepening. Each new "state-of-the-art" model demands disproportionately more compute, memory, and data. Diminishing returns are already visible. Recent reports from Bloomberg and The Information indicate that OpenAI, Google, and Anthropic are experiencing slower improvement rates despite massive investments in computing power and data.[12] Research from Epoch AI shows that training costs for frontier models have grown by a factor of 2-3x per year over the past eight years, with today's most expensive models costing tens of millions of dollars to train—suggesting costs could exceed $1 billion by 2027.[13]

Future breakthroughs may depend less on bigger models and more on smarter architectures, efficiency gains, and hybrid reasoning systems that combine symbolic, causal, and neural elements.

Tentative Conclusions

Financially, AI may contain pockets of speculative excess, but the underlying infrastructure is real and strategically important. A correction, if it comes, will likely prune hype rather than kill the field.

Technically, AI delivers extraordinary capabilities in text-based reasoning and automation but remains narrow. General intelligence, common sense, and real-world generalisation remain unsolved.

Agentic AI could evolve into the backbone of digital operations—the next generation of workflow orchestration rather than an entirely new paradigm.

Sustainability will become the next frontier constraint: compute, power, and cooling capacity may shape AI's growth curve more than algorithms.

In short, AI is not a mirage—but neither is it omnipotent. It's a technological wave still searching for equilibrium between hype, utility, and feasibility.

"We may be living through both a bubble and a revolution—the two are not mutually exclusive."

References

[1]: Bloomberg. (October 2025). "AI Is Dominating 2025 VC Investing, Pulling in $192.7 Billion." https://www.bloomberg.com/news/articles/2025-10-03/ai-is-dominating-2025-vc-investing-pulling-in-192-7-billion

[2]: Crunchbase. (October 2025). "Q3 Venture Funding Jumps 38% As More Massive Rounds Go To AI Giants." https://news.crunchbase.com/venture/global-vc-funding-biggest-deals-q3-2025-ai-ma-data/

[3]: WebProNews. (October 2025). "Microsoft, Alphabet, Amazon Drive AI Supercycle with $364B Investments." https://www.webpronews.com/microsoft-alphabet-amazon-drive-ai-supercycle-with-364b-investments/; Microsoft On the Issues. (January 2025). "The golden opportunity for American AI." https://blogs.microsoft.com/on-the-issues/2025/01/03/the-golden-opportunity-for-american-ai/

[4] IBM Think. (April 2025). "How smart is machine intelligence? AI aces games but fails basic reality check." https://www.ibm.com/think/news/mit-study-evaluating-world-model-ai; MLQ.ai. (2025). "Apple Research Exposes Limits of AI Reasoning Models." https://mlq.ai/news/apple-research-exposes-limits-of-ai-reasoning-models-ahead-of-wwdc-2025/; Misra, V. (June 2025). "The Illusion of Thinking: Why Language Models Can't Improve Themselves." https://medium.com/@vishalmisra/the-illusion-of-thinking-why-language-models-cant-improve-themselves-0c71a13811e2

[5]: International Energy Agency. (2024). "Data centres & networks." https://www.iea.org/energy-system/buildings/data-centres-and-data-transmission-networks

[6]: Nature. (April 2025). "Data centres will use twice as much energy by 2030—driven by AI." https://www.nature.com/articles/d41586-025-01113-z

[7]: International Energy Agency. (2024). "Energy demand from AI." https://www.iea.org/reports/energy-and-ai/energy-demand-from-ai

[8]: Sapient Intelligence. (June 2025). "Hierarchical Reasoning Model." https://arxiv.org/abs/2506.21734; ARC Prize. (2025). "The Hidden Drivers of HRM's Performance on ARC-AGI." https://arcprize.org/blog/hrm-analysis

[9]: Kojima, T., et al. (2022). "Large Language Models are Zero-Shot Reasoners." https://arxiv.org/abs/2205.11916

[10]: Wei, J., et al. (2022). "Emergent Abilities of Large Language Models." https://arxiv.org/abs/2206.07682; Center for Security and Emerging Technology. (April 2024). "Emergent Abilities in Large Language Models: An Explainer." https://cset.georgetown.edu/article/emergent-abilities-in-large-language-models-an-explainer/

[12]: Marketing AI Institute. (November 2024). "The Great AI Scaling Debate: Have We Hit a Wall?" https://www.marketingaiinstitute.com/blog/scaling-laws-ai-wall; Platformer. (November 2024). "AI companies hit a scaling wall." https://www.platformer.news/openai-google-scaling-laws-anthropic-ai/

[13]: Epoch AI. (January 2025). "How much does it cost to train frontier AI models?" https://epoch.ai/blog/how-much-does-it-cost-to-train-frontier-ai-models; Sevilla, J., et al. (May 2024). "The rising costs of training frontier AI models." https://arxiv.org/abs/2405.21015

[14]: n8n. (2025). "Advanced AI Workflow Automation Software & Tools." https://n8n.io/ai/; Hostinger. (September 2025). "What is n8n? Intro to a workflow automation tool." https://www.hostinger.com/tutorials/what-is-n8n

[15]: Gupta, M. (October 2025). "OpenAI AgentKit vs N8N: The best AI Workflow Builder?" https://medium.com/data-science-in-your-pocket/openai-agentkit-vs-n8n-the-best-ai-workflow-builder-da5eaf21aa10

It may seem long time ago when companies such as Dropbox, which famously saved nearly $75 million by migrating away from public cloud infrastructure, and 37signals (creators of Basecamp), who documented their strategic move back to on-premises infrastructure, brought critical attention to the often-overlooked complexities of cloud economics. These case studies reveal that uncontrolled cloud adoption without meticulous cost management can lead to spiralling expenses that erode the very benefits organisations seek.

However, abandoning cloud strategies wholesale is not the answer. Instead, in this blog post I will delve into practical, actionable strategies for Azure cost optimisation that can help organisations maintain the agility of cloud computing while keeping expenditures firmly in check. From right-sizing resources to leveraging advanced cost management tools, I will explore how organisations can transform cloud spending from a potential financial burden into a strategic advantage.

Cloud Repatriation: When Cloud Does not Deliver

The narrative of cloud computing is not a simple tale of universal triumph. While cloud migration has been touted as an inevitable technological progression, some of the tech industry's most innovative companies found themselves swimming against the current, choosing to bring their infrastructure back in-house. I will briefly touch on the cases of Dropbox and Basecamp as these are the ones that have been covered widely.

Dropbox: A Strategic Infrastructure Exodus

Dropbox's cloud repatriation journey is perhaps the most celebrated case of calculated infrastructure transformation. In 2016, the file-sharing giant made a bold decision to migrate approximately 90% of its infrastructure from Amazon Web Services (AWS) to a custom-built, private cloud infrastructure. This move was driven by a number of factors including having control to configure hardware precisely, performance, and pure economic pragmatism.

The financial implications were staggering. By building their own infrastructure, Dropbox saved approximately $75 million over two years. Their custom-built infrastructure provided remarkable advantages, allowing the company to precisely configure hardware tailored to their specific workloads. This approach enabled significant reduction in per-unit infrastructure costs, offering greater control over performance and resource allocation while eliminating the cloud provider markup on infrastructure services.

The key challenges Dropbox faced with public cloud were multifaceted. The company experienced escalating costs that grew disproportionately with their scale. Standard cloud offerings provided limited customisation options, and the performance overhead associated with multi-tenant cloud environments became increasingly problematic as the company expanded.

Basecamp: Embracing Infrastructure Autonomy

David Heinemeier Hansson, CTO of 37signals (Basecamp), has been vocally critical of the cloud-first approach. Their repatriation strategy was driven by a combination of cost considerations and a philosophical stance on infrastructure ownership.

Basecamp's migration illuminated several critical issues with cloud computing. The company struggled with unpredictable and rapidly escalating cloud costs, experiencing diminishing returns on cloud flexibility for their stable, well-understood workloads. They discovered that direct hardware management could provide superior performance and cost-efficiency compared to cloud-based solutions.

By moving back to dedicated hardware, Basecamp achieved a transformative infrastructure strategy. The company secured more predictable infrastructure expenses, enhanced performance through direct hardware control, reduced complexity in infrastructure management, and gained greater long-term cost predictability.

Common Reasons for Cloud Repatriation

While Dropbox and Basecamp represent different scales and approaches, their experiences reveal common challenges in cloud migration. Cost scalability emerged as a critical concern, with cloud expenses potentially growing exponentially and outpacing the perceived benefits of flexibility. Performance limitations became apparent, demonstrating that generic cloud infrastructure does not always match the efficiency of custom-built solutions.

Each organisation discovered an economic tipping point where owning infrastructure becomes more economical than renting. It is crucial to understand that cloud repatriation is not a universal solution. These companies did not abandon cloud computing entirely but rather made strategic decisions about where and how to deploy their computational resources.

The lesson is not that cloud is inherently flawed, but that a one-size-fits-all approach to cloud infrastructure is fundamentally misguided. Successful digital infrastructure strategy requires continuous evaluation, flexibility, and a willingness to challenge prevailing technological narratives.

Azure Cost Optimisation Strategies

Understanding the cautionary tales of cloud repatriation does not mean abandoning cloud strategies altogether. Instead, it calls for a more nuanced, strategic approach to cloud cost management. Microsoft Azure offers a robust ecosystem of tools and techniques that, when implemented thoughtfully, can transform cloud expenditure from a potential financial drain into a strategic business advantage.

Cloud cost optimisation is not a one-time exercise but a continuous process of analysis, refinement, and strategic alignment. It requires organisations to develop a holistic view of their cloud infrastructure, moving beyond simple cost-cutting to create a more intelligent, efficient computational environment. The most successful organisations approach Azure cost management as a dynamic discipline that balances performance, scalability, and financial prudence.

In this section, I will explore a suite of strategies that can help organisations extract maximum value from their Azure investments. From granular resource management to advanced cost prediction techniques, these approaches will empower IT leaders and financial managers to take control of their cloud economics. The goal is not just to reduce costs, but to create a more responsive, adaptable, and financially sustainable cloud infrastructure that directly supports business objectives.

1. Right-Sizing Resources

Right-sizing represents one of the most fundamental yet powerful strategies for optimising Azure infrastructure costs. At its core, right-sizing is about matching computational resources precisely to workload requirements, eliminating the costly practice of over-provisioning that plagues many organisation cloud environments.

Most organisations inadvertently deploy virtual machines and cloud resources with excessive capacity, essentially paying for computational power they never utilise. Industry studies suggest that many organisations waste up to 35% of their cloud spending on unused or overprovisioned resources. Any engineer who has deployed on-premises knows that when requesting for a VM, you ask for the highest specifications you can get away with because it is almost impossible, mostly due to paperwork involved, to request an upgrade to your on-prem VM (let alone a physical machine) at a later stage. Azure provides sophisticated tools that enable organisations to analyse resource utilisation with remarkable granularity, transforming cloud cost management from a guessing game to a data-driven discipline.

The right-sizing process begins with comprehensive monitoring and analysis. Azure Monitor and Azure Advisor become critical allies in this journey, offering detailed insights into resource consumption patterns. These tools track critical metrics such as CPU utilisation, memory consumption, network throughput, and storage performance across virtual machines and cloud services.

For virtual machines, right-sizing strategies can be divided into several approaches. Downsizing involves reducing virtual machine specifications to match actual workload requirements. This might mean transitioning from a premium D-series virtual machine with 16 cores to a more modest machine with 4 cores that can adequately handle the computational load. Similarly, organisations can leverage Azure's burstable virtual machine instances, which provide baseline performance with the ability to burst above that baseline when required, offering significant cost savings.

Reserved Instances represent another sophisticated right-sizing mechanism. By committing to one-year or three-year terms for specific virtual machine configurations, organisations can secure substantial discounts compared to pay-as-you-go pricing. These reservations work exceptionally well for stable, predictable workloads where computational requirements remain relatively consistent. This option, however, is not suitable for everyone; it is mostly suitable for large organisations.

The economic implications are profound. A well-executed right-sizing strategy can potentially reduce cloud infrastructure costs by 30-50% without compromising performance or introducing additional complexity. However, right-sizing is not a one-time exercise but a continuous process requiring regular review and adjustment.

Key considerations for effective right-sizing include:

• Implementing continuous monitoring of resource utilisation

• Establishing clear performance baseline metrics

• Creating automated scaling policies

• Regularly reviewing and adjusting resource allocations

• Leveraging Azure's native cost management and recommendation tools

Organisations must also develop a nuanced understanding of their workload characteristics. Some applications require consistent computational power, while others experience significant variability. Right-sizing strategies must be tailored to these unique workload profiles, recognising that a universal approach is fundamentally ineffective.

Technical teams, while they should not become finance managers, should understand the cost implications of the resources they are deploying. This approach ensures that computational resources are not just efficient, but directly aligned with broader organisational objectives.

2. Advanced Cost Management Techniques

Azure Cost Management and billing tools serve as the control centre for enterprise cloud spending. These native tools provide comprehensive visibility into resource consumption patterns and spending trends across subscriptions and resource groups. The platform's cost analysis features enable organisations to break down expenditure by service, location, and time period, while forecasting capabilities help predict future spending based on historical patterns.

Budget alerts and spending limits act as an early warning system for potential cost overruns. Organisations can establish multiple budget thresholds with automated notifications at different spending levels. When configured effectively, these alerts notify stakeholders via email when spending reaches predefined percentages of the budget, typically at 70%, 90%, and 100%. Critical workloads can be protected by implementing hard spending limits that automatically disable resource deployment when budgets are exceeded.

A well-structured tagging strategy enables granular cost tracking and allocation. Tags should reflect business dimensions such as department, environment, application, and cost centre. This granular approach to resource labelling enables precise cost attribution and chargeback mechanisms. Through consistent tagging, organisations can generate detailed reports showing exactly how cloud resources are being consumed across different business units and projects. Effective tagging also facilitates automation of cost management policies and governance rules, ensuring resources are consistently tracked and managed according to organisational standards.

Integration of these three approaches creates a robust framework for cost governance. Real-time visibility, proactive alerts, and detailed tracking mechanisms work together to prevent unexpected costs while maintaining operational efficiency.

3. Storage Optimisation

Storage costs in Azure can be effectively managed through a multi-layered optimisation strategy. Azure Blob Storage access tiers form the foundation of cost optimisation, offering Hot, Cool, Cold, and Archive tiers with decreasing storage costs but increasing access fees. Organisations should implement lifecycle management policies to automatically move data between these tiers based on access patterns and retention requirements.

Data redundancy choices significantly impact costs. While Geo-Redundant Storage (GRS) provides the highest durability, many workloads can safely utilise Locally Redundant Storage (LRS) or Zone-Redundant Storage (ZRS) at a lower cost. Organisations should evaluate their Recovery Point Objectives (RPO) and adjust redundancy accordingly.

Implementing effective data retention policies helps control storage growth. Regular clean up of unused snapshots, old backups, and obsolete data can substantially reduce storage costs. Azure Storage Explorer enables identification of orphaned resources and unnecessary duplicates.

Compression and deduplication techniques further optimise storage usage. Implementing these at the application level before storing data in Azure can significantly reduce storage requirements and associated costs.

Premium storage should be reserved for truly I/O-intensive workloads, while standard storage suffices for most general-purpose applications. Regular monitoring of storage metrics helps identify opportunities to downgrade storage performance tiers without impacting application performance.

Another component of storage is databases. Azure offers a few options to choose from, including MS SQL, PostgreSQL, Cosmos, and other open-source databases such as MongoDB. It is tempting for engineers to choose the hottest technology at any point in time. However, this may be detrimental to cost management. For example, while Cosmos DB offers features such as instant multi-region replication, multi-write replicated copies, and sub-millisecond reads, as well as APIs for MongoDB, SQL, etc., it comes at a cost. Organisations need to analyse their requirements and understand whether the additional benefits offered by Cosmos DB are truly what their services require.

4. Network Cost Reduction

Network costs in Azure often form a substantial portion of cloud spending, particularly for data-intensive applications. Effective network design starts with proper Virtual Network (VNet) architecture and region selection. By hosting interdependent services within the same region and availability zone, organizations can minimize inter-region data transfer costs while maintaining high availability.

Azure ExpressRoute proves cost-effective for organisations with high-volume data transfer requirements between on-premises and Azure environments. While initial setup costs are higher than VPN connections, ExpressRoute's predictable pricing and superior performance often result in long-term savings for large-scale deployments.

Content Delivery Network (CDN) implementation significantly reduces data transfer costs for globally distributed applications. Azure CDN caches content closer to end users, reducing both latency and egress charges. Similarly, careful placement of Azure Front Door and Application Gateway services optimises traffic routing and reduces unnecessary data transfer.

Network bandwidth costs can be controlled through effective use of Azure's Virtual Network service endpoints. These endpoints allow services to connect through Azure's backbone network rather than through public IP addresses, reducing both costs and security risks.

Data egress optimisation requires careful monitoring of cross-region and internet-bound traffic. Azure Network Watcher and Flow Logs provide visibility into traffic patterns, enabling identification of unexpected or costly data transfers. Regular review of these metrics helps identify opportunities for traffic optimisation and potential cost savings through architectural improvements.

Implementing bandwidth throttling and scheduling large data transfers during off-peak hours can help manage costs while maintaining service quality. Additionally, utilising compression for data transfers and implementing efficient caching strategies at the application level reduces overall network utilisation.

5. Containerisation and Serverless

Containerisation through Azure Kubernetes Service (AKS) and serverless computing via Azure Functions represent modern approaches to cost optimisation. AKS enables efficient resource utilisation through dynamic container orchestration, automatically scaling resources based on actual demand. This eliminates idle capacity costs while ensuring applications receive necessary resources during peak periods.

Azure Functions' consumption plan pricing model charges only for actual execution time and memory usage, measured in milliseconds. This granular pricing eliminates the overhead costs associated with maintaining traditional infrastructure. Functions automatically scale based on workload, optimising costs during varying demand levels.

Containerisation facilitates efficient resource sharing among applications, improving overall infrastructure utilisation. Containers' lightweight nature allows higher density deployment compared to traditional virtual machines, reducing per-application infrastructure costs. AKS's automated bin-packing capabilities ensure optimal resource distribution across the cluster.

Serverless architecture eliminates infrastructure management costs and reduces development overhead. Azure Functions' integration with other Azure services enables cost-effective event-driven architectures. The platform handles scaling, patching, and maintenance, reducing operational expenses.

Combined implementation of containers and serverless creates a hybrid approach optimised for different workload types. Long-running applications benefit from container-based deployment, while event-driven processes leverage serverless functions. This architecture maximises cost efficiency while maintaining application performance and scalability.

Both technologies support rapid deployment and testing, reducing development costs and time-to-market. Integration with Azure DevOps enables automated deployment pipelines, further optimising operational efficiency and resource utilisation.

Containerisation and serverless means that organisations need to assess their on-prem workloads and find the best way how these workloads can take advantage of cloud native infrastructure. This may mean redesigning the workloads to best suit the cloud infrastructure. For start-ups, this is never a problem as there is no on-prem workloads to migrate.

Practical Recommendations

1. Conduct Regular Cost Audits

   Regular cost audits form the backbone of effective cloud financial management. Organisations should establish a systematic process to review Azure spending patterns monthly, focusing on identifying unexpected cost spikes and underutilised resources. These audits should examine resource utilisation across all subscriptions, comparing actual spending against budgeted amounts and historical trends.

   Cost audits should integrate data from Azure Cost Management, emphasising high-impact areas such as virtual machines, storage, and network usage. The process should identify orphaned resources, non-production environments running outside business hours, and opportunities for reservation purchases. Organisations should also analyse usage patterns to detect potential cost anomalies or security incidents that manifest as unusual spending patterns. A comprehensive audit includes reviewing tag compliance, ensuring accurate cost allocation across business units, and validating that resources align with governance policies. The findings should drive actionable recommendations for immediate cost optimisation and long-term architectural improvements.

2. Develop a Hybrid Strategy

   Developing a hybrid strategy is essential for optimising cloud infrastructure. Not all workloads are suited for a cloud-native environment, making it crucial to evaluate the specific needs of each application. A hybrid approach allows organisations to leverage the strengths of both cloud and on-premises solutions, ensuring that workloads are matched to the most appropriate infrastructure. This strategy involves integrating cloud services with existing on-premises systems, providing flexibility and scalability while maintaining control over critical data and applications. By adopting a multi-cloud or hybrid model, organisations can avoid vendor lock-in, enhance resilience, and optimise costs. This approach requires careful planning and execution, ensuring seamless interoperability between different environments. Organisations should continuously assess their infrastructure needs, adapting their strategy to align with evolving business objectives and technological advancements. A well-executed hybrid strategy not only optimises performance and cost-efficiency but also supports long-term growth and innovation.

3. Invest in Cloud Financial Management

   Investing in cloud financial management is crucial for effective cost optimisation. Organisations should prioritise training teams in cloud economics to ensure a comprehensive understanding of cost dynamics. Establishing cross-functional FinOps teams can bridge the gap between finance and technology, fostering collaboration and informed decision-making. Developing clear cloud spending policies is essential to guide resource allocation and cost control. These policies should be aligned with organisational objectives, ensuring that cloud investments support business goals. Continuous monitoring and analysis of cloud expenditure enable proactive management, allowing organisations to identify cost-saving opportunities and address inefficiencies promptly. By integrating financial management practices into cloud operations, organisations can achieve greater transparency and accountability in their spending. This strategic approach not only optimises costs but also enhances the overall value derived from cloud investments, supporting sustainable growth and innovation in a competitive digital landscape.

Emerging Trends

Emerging trends in cloud cost optimisation are reshaping how organisations approach their digital infrastructure. The rise of FinOps, a practice that combines financial accountability with cloud operations, is gaining traction as companies seek to align their cloud spending with business objectives. This approach encourages collaboration between finance, technology, and business teams, fostering a culture of cost awareness and strategic investment. Additionally, advancements in artificial intelligence and machine learning are enabling more sophisticated cost management tools. These technologies provide predictive analytics and automated recommendations, allowing organisations to anticipate cost fluctuations and optimise resource allocation proactively. The growing emphasis on sustainability is also influencing cloud strategies, with organisations seeking to reduce their carbon footprint through efficient resource utilisation and green cloud initiatives. As cloud providers enhance their offerings with energy-efficient infrastructure and carbon tracking capabilities, organisations are increasingly considering environmental impact alongside financial metrics. These emerging trends underscore the importance of a holistic approach to cloud cost optimisation, where financial prudence, technological innovation, and sustainability converge to drive long-term value. By staying attuned to these developments, organisations can navigate the complexities of cloud economics and harness the full potential of their digital investments.

Conclusion

In conclusion, while the allure of cloud migration is undeniable, it is not a solution for all infrastructure challenges. Successful Azure cost optimisation requires a strategic approach that balances the benefits of cloud flexibility with the need for financial prudence. By implementing continuous monitoring, strategic resource allocation, and flexible infrastructure planning, organisations can transform cloud spending from a potential financial burden into a strategic advantage. By learning from the experiences of companies like Dropbox and Basecamp, and embracing robust cost management techniques, organisations can harness the full potential of Azure, ensuring that their cloud investments are both economically and operationally sound.

References

[1] Dropbox Engineering Blog, "Scaling to Exabytes and Beyond" (2016)

[2] David Heinemeier Hansson, "Why we're leaving the cloud" (2022)

The Evolution from Engineer to Leader

The transition from a hands-on engineer to a technical leader is often fraught with challenges. Many engineers are promoted due to their technical excellence, only to realise that the skills that made them exceptional engineers do not necessarily translate into effective leadership. As an engineer, you spend years perfecting your craft, whether it's designing, implementing, or maintaining scalable systems. However, upon entering a leadership role, your world shifts dramatically - your deliverables are no longer solely dependent on your efforts. You must rely on others to achieve your Objective Key Results (OKRs). How do you navigate this new landscape? An engineer’s initial instinct might be, "I know my craft, and I don't trust others to do as good a job as I can. I'll just do it myself." Those who are more daring may allow others to contribute but will meticulously check and recheck everything to ensure it meets their standards. Neither approach is sustainable, and some engineers may eventually conclude that leadership is not for them, preferring to remain individual contributors. But who is responsible when engineers find themselves in leadership roles for which they are unprepared? It's perfectly reasonable for an engineer to choose to remain an individual contributor, as long as it's an informed decision with a clear understanding of the alternative. My hypothesis is that when leadership focuses on motivating and inspiring people, engineers can make well-informed decisions. Leaders who possess more than just technical skills have the ability to effectively support their teams.

Essential Non-Technical Leadership Skills

“I have delivered on the objectives we agreed upon at the beginning of the year, and I expected to be on the list of promoted employees, but I am not. What happened?” asks an engineer. The manager replies, “But I didn't know you wanted a promotion.” This conversation highlights a significant disconnect between the engineer and the manager. How did it reach this point? The engineer assumed that meeting objectives would naturally lead to a promotion, while the manager was unaware of the engineer's aspirations.

This scenario underscores the importance of clear communication and expectation management within teams. It is crucial for both engineers and managers to engage in open dialogues about career goals and development plans. Regular check-ins and feedback sessions can help bridge the gap, ensuring that both parties are aligned on expectations and aspirations.

Moreover, this situation illustrates the need for managers to develop essential leadership skills beyond technical expertise. Effective leaders must actively listen, understand their team members' career ambitions, and provide guidance and support to help them achieve their goals. By fostering an environment of transparency and mutual understanding, leaders can cultivate a motivated and engaged team.

In the following sections, I will discuss non-technical skills necessary for leading successful technical teams. These skills help prevent misunderstandings, promote a culture of growth and development, ensure organizational goals are met, and support the development of future leaders.

1. Emotional Intelligence and Active Listening

   The ability to truly listen and understand team members' perspectives is fundamental to effective leadership. Active listening means deliberately being present during one-on-ones, giving your direct the space and time to express their lows and highs and aspirations. It also means reading between the lines when team members express concerns directly or indirectly while acknowledging emotions without immediately jumping to solutions. For example, when an engineer says, "This deadline seems unrealistic," a good leader doesn't immediately defend the timeline but might respond, "Tell me more about your concerns. What specific challenges do you foresee?" Conversely, an engineer might say “it's fine, I will deliver this by the deadline” when they are effectively saying I will work late nights and weekends to get this done. By applying emotional intelligence and active listening, a leader will read between the lines and get the true meaning of what their direct report means.

2. Understanding and Supporting Neurodiversity

   Modern tech teams are increasingly diverse, including team members who may be neurodivergent. Effective leaders treat each member of their team as unique individuals and, therefore, strive to understand what differentiates each member from others. It does not mean asking intrusive questions but, by practicing emotional intelligence and active listening, one can understand individual members better. When leaders use their one-to-one as a safe space platform for their team to discuss what matters to them, they have a chance at understanding them better.

   The multitude of communication channels that currently exist provide both opportunities and challenges. When you understand your team, as a leader you should provide multiple communication channels (chat, email, face-to-face) to accommodate different preferences. You should also create clear and explicit expectations. Where possible, offer flexible work environments that support different sensory needs. Moreover, aim to apply various modes when sharing information - this could be a combination of visual, audio and text.

3. Strategic Communication

   There are few cases where technical solutions do not have non-technical stakeholders. It is, therefore, critical that technical leaders can bridge the gap between technical and non-technical stakeholders. This involves translating complex technical concepts into business value and vice versa. Communication must, therefore, be adapted for different audiences. One way of doing so is by using storytelling to make technical information more accessible. Instead of telling your business stakeholders "We need to refactor our monolithic architecture into microservices," an effective leader might say, "We can reduce our time-to-market by 40% and cut deployment risks in half by modernising our software architecture."

4. Coaching and Mentorship

   Great technical leaders develop their team members through regular feedback sessions focused on growth and creating individualised development plans. It also involves identifying and nurturing both technical and leadership potential. As a technical leader, coaching and mentorship are critical responsibilities that go far beyond simply managing tasks. These practices involve deliberately investing in the professional and personal growth of team members, creating an environment of continuous learning and development.

   Effective coaching means providing targeted guidance that helps individuals identify their strengths and address developmental areas. This is not about dictating solutions, but asking probing questions that enable team members to discover insights independently. A technical leader coaches by offering constructive feedback, helping team members troubleshoot complex problems.

   Mentorship, meanwhile, takes a broader, more holistic view of an individual's career trajectory. It involves sharing personal experiences, industry insights, and strategic career advice. A technical mentor helps junior professionals navigate technical challenges, understand unwritten organisational dynamics, and develop both technical and soft skills critical for long-term success.

5. Conflict Resolution and Difficult Conversations

   Both technical and non-technical disagreements can become personal without proper leadership. Effective leaders focus on facts and impact rather than blame. They create safe spaces for healthy debate and guide teams toward consensus while ensuring all voices are heard. For instance, when two senior engineers disagree on a technical approach, a good leader might structure a decision-making framework that evaluates options against agreed-upon criteria, turning a potential conflict into a collaborative problem-solving exercise.

6. Understanding Diverse Perspectives

   Tech teams are often composed of individuals from diverse backgrounds and experiences. As a leader, understanding diverse perspectives is fundamental to creating an inclusive, innovative, and high-performing environment. This goes beyond mere tolerance - it requires active listening, genuine curiosity, and a commitment to creating psychological safety where all team members feel valued and heard.

   Diverse perspectives bring richness to problem-solving by introducing varied insights, challenging assumptions, and uncovering blind spots. A leader must intentionally create spaces for different voices to emerge, whether through structured feedback mechanisms, or inclusive meeting formats.

   The key is to move from passive acknowledgment to active integration. This means not just hearing different viewpoints, but actively seeking them out, creating mechanisms for their expression, and demonstrating how these perspectives directly influence decision-making. When team members see their unique contributions genuinely respected and incorporated, it fosters trust, engagement, and collective ownership.

7. Recognising and Addressing Emotional Needs

   As a leader, recognising and addressing emotional needs is a critical skill that transcends traditional management approaches. It requires deep emotional intelligence and the ability to create a supportive environment where team members feel psychologically safe and understood.

   Effective leaders acknowledge that emotions are not separate from professional performance, but integral to it. They create space for authentic conversations, actively listen without judgment, and demonstrate empathy. This means recognising signs of stress, burnout, or personal challenges, and responding with compassion and practical support.

   The goal is not to become a therapist, but to foster a human-centred workplace where individuals feel valued beyond their productivity. This involves checking in genuinely about team members' well-being, offering flexible support during challenging times, creating mechanisms for open, non-punitive communication, and modelling emotional vulnerability and resilience. By prioritising emotional intelligence, leaders build trust, enhance team cohesion, and create an environment where individuals can bring their whole, authentic selves to work.

8. Fostering Talent

   As a leader, fostering talent is a strategy that goes beyond traditional performance management. It's about creating an ecosystem where individual potential can be discovered, nurtured, and fully realised.

   Effective talent cultivation requires a multifaceted approach. This means providing clear developmental pathways, offering challenging assignments that stretch capabilities, and creating opportunities for continuous learning. Leaders must act as talent architects, designing personalised growth plans that align individual aspirations with organisational goals.

   Key strategies to talent cultivation include conducting regular, meaningful career development conversations, providing targeted mentorship and coaching, creating exposure opportunities, investing in skill development programs, and recognising and rewarding potential, not just current performance.

Practical Steps for Developing Non-Technical Leadership Skills

1. Seek Feedback Actively

   Seeking feedback actively is a transformative leadership practice that requires genuine vulnerability and commitment to personal growth. Effective leaders create multiple channels for receiving candid, constructive input from peers, team members, and mentors. This means establishing regular, structured feedback mechanisms like 360-degree reviews, informal check-ins, and open communication platforms that encourage honest dialogue.

   The key is to approach feedback not as a defensive exercise, but as a strategic opportunity for development. Leaders must demonstrate openness by actively listening, asking clarifying questions, and visibly implementing insights gained. This involves setting aside ego, embracing potential areas of improvement, and modelling a growth mindset that encourages continuous learning. By consistently seeking and acting on feedback, leaders signal their commitment to personal and organisational improvement, creating a culture of transparency, trust, and ongoing professional development.

2. Find a Leadership Mentor

   Finding a leadership mentor is a strategic investment in personal and professional development. Successful leaders actively seek mentors who can provide nuanced guidance, share strategic insights, and offer perspectives gained from extensive leadership experience. This relationship goes beyond traditional coaching, creating a trusted partnership where wisdom is transferred through deep, reflective conversations.

   The most effective mentorship connections are not formalised transactions, but organic relationships built on mutual respect, shared values, and genuine curiosity. Leaders should seek mentors who challenge their thinking, expose them to different leadership approaches, and provide honest, constructive feedback.

3. Practice Deliberately

   Practicing deliberately is a methodical approach to leadership skill development that requires intentional, focused effort and continuous self-reflection. Unlike casual learning, deliberate practice involves setting specific leadership development goals, creating structured scenarios to challenge existing capabilities, and systematically analysing performance and outcomes.

   Deliberate practice demands vulnerability - embracing discomfort, acknowledging limitations, and treating each leadership experience as a learning opportunity. By maintaining a disciplined, self-aware approach to skill development, leaders can progressively expand their capabilities, transforming potential into tangible leadership expertise through consistent, intentional effort.

4. Invest in Learning

   Investing in learning is a critical leadership strategy that goes beyond traditional training programs. Successful leaders view continuous learning as a strategic imperative, allocating time, resources, and mental energy to expanding their knowledge and skills. This means actively seeking diverse learning opportunities, from formal educational programs and workshops to reading, podcasts, and cross-functional experiences.

   Effective learning investment involves creating a personalised development plan that aligns with both personal growth objectives and organisational needs. Leaders must be intentional about exploring areas outside their comfort zone, embracing interdisciplinary knowledge that can provide fresh perspectives on leadership challenges.

   Effective leaders cultivate a growth mindset that sees learning as a lifelong journey, continuously challenging themselves, remaining curious, and demonstrating a commitment to personal and professional development that inspires their teams.

The Impact of Strong Leadership

When technical leaders develop these non-technical skills, the results are significant. Strong leadership represents a powerful fusion of technical expertise and profound interpersonal capabilities that fundamentally transforms organisational potential. Technical skills provide the foundational knowledge and strategic understanding necessary to navigate complex technical environments, while soft skills enable the translation of that knowledge into meaningful, collaborative action.

A truly impactful leader creates an ecosystem where technical proficiency and human-centred approaches coexist harmoniously. This means understanding intricate technical challenges while simultaneously cultivating an environment of psychological safety, trust, and continuous learning. Technical skills ensure strategic direction and operational excellence, while soft skills like emotional intelligence, communication, and empathy enable leaders to inspire, motivate, and align diverse teams toward shared objectives.

The most transformative leaders recognise that technical competence alone is insufficient. They build bridges between complex technical concepts and human potential, creating narratives that make strategic goals meaningful and engaging. They translate technical challenges into opportunities for growth, innovation, and collective achievement.

Conclusion

While technical expertise might get you into a leadership position, it is the mastery of non-technical skills that will determine your success as a leader. The most respected technical leaders are those who can balance technical knowledge with strong people skills, creating environments where both technology and people can thrive. Leadership is a journey, not a destination. The most successful technical leaders are those who remain committed to growing both their technical and non-technical capabilities throughout their careers.

The Rise of Azure Container Apps

Azure Container Apps, introduced in 2021, represent Microsoft's serverless container offering. It provides a simplified approach to deploying containerised applications without the complexity of managing a Kubernetes cluster. Think of it as a middle ground between Azure Functions and AKS. It is a fully managed serverless container platform that simplifies application deployment and scaling. It eliminates the need for managing infrastructure or Kubernetes clusters, allowing developers to focus on building and deploying applications.

Key Features of Container Apps:

• Serverless: Automatic scaling based on HTTP traffic, events, or Kubernetes Event-Driven Autoscaling (KEDA) supported triggers.

• Built-in Service Mesh: Features such as Distributed Application Runtime (Dapr) make it easier to implement state management, message brokering, and service-to-service communication.

• Built-in service discovery and ingress: each container app in your environment can communicate with others using their DNS name. There’s no need to configure or manage complex networking settings manually.

• Fully managed: Azure handles infrastructure provisioning, scaling, and security.

• Simplicity: Developers do not need to understand Kubernetes internals to deploy and manage applications.

• Pay-per-use pricing model: You can scale your services to zero instances and thereby save during periods of no activity in your applications.

Azure Kubernetes Service: Enterprise-Grade Container Orchestration

AKS is a fully managed Kubernetes service that provides a platform for deploying and managing containerised applications. It simplifies the complexities of Kubernetes by handling infrastructure provisioning, cluster management, and security. AKS is well-suited for a wide range of applications, from simple microservices to complex, stateful workloads.

Despite AKS being a managed service, Microsoft only manages the control plane, underlying infrastructure and security. Engineers, however, are responsible for a number of components such as:

• Node Pool Management:

  • Engineers are responsible for configuring node pools, including VM sizes, auto-scaling, and labels for workload segregation.

  • They also need to ensure the underlying worker nodes (VMs) have sufficient capacity to run workloads efficiently.

• Cluster Networking:

  • Setting up appropriate network models (Azure CNI or Kubenet) and configuring pod-to-pod and pod-to-service communication.

  • Implementing ingress controllers (e.g., NGINX, Application Gateway) and setting up DNS.

  • Managing IP address ranges, subnets, and custom VNET configurations.

Despite the simplicity of Container Apps, AKS remains a powerhouse for complex, enterprise-scale container orchestration. It offers complete control over your Kubernetes environment while abstracting away the control plane management.

Key Features of AKS:

• Full Kubernetes API access: AKS gives you the full power of Kubernetes, allowing custom configurations and integrations with third-party tools.

• Granular Control: Developers and devops teams can fine-tune deployment strategies, manage scaling policies, and implement advanced networking and security configurations.

• Multi-Container Support: Ideal for microservices architectures that require inter-container communication and advanced workflows.

• Horizontal scaling: Easily scale applications up or down to meet changing demands.

• Support for stateful applications: Supports persistent storage for stateful workloads like databases or message queues.

Key Differences from a deployment perspective

• Development Complexity

# Container Apps
# container-apps.yaml
resources:
  containerApps:
    # name of the container app
    name: my-cool-app
    containers:
      #container to be deployed to this container app
      - image: myregistry.azurecr.io/mycoolapp:v1
        env:
          #environment variables to be passed to the container at runtime
          - name: PORT
            value: "80"
# Additional configuration required...

This YAML configuration defines an Azure Container App deployment, specifying a container named "my-cool-app" that will use an image from an Azure Container Registry (myregistry.azurecr.io/mycoolapp:v1). The configuration sets up a basic container deployment with a single environment variable (PORT set to 80), indicating how the container should be initialised and run. While this snippet provides the fundamental structure for deploying a containerised application, it represents only a partial configuration and would require additional settings for a complete container app deployment in an Azure environment. The snippet is only used for illustration.

#AKS
# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-cool-app
spec:
  #3 pods for this deployment
  replicas: 3
  selector:
    #must match the label defined under template
    matchLabels:
      app: my-cool-pp
  template:
    metadata:
      labels:
        app: my-cool-app
    spec:
      containers:
      #one container specified
      - name: my-cool-app
        image: myregistry.azurecr.io/mycoolapp:v1
        env:
        - name: PORT
          value: "80"

#required to create a Kubernetes Service, which exposes 
#the pods created by the Deployment.
# service.yaml

apiVersion: v1
kind: Service
metadata:
  name: my-cool-app-service
spec:
  selector:
    app: my-cool-app
  ports:
    - protocol: TCP
      port: 80    # Port exposed by the Service
      targetPort: 80 # Port the container listens on
  type: ClusterIP # Internal access within the cluster
# Additional configuration required...

This Kubernetes configuration defines a deployment and service for an application in Azure Kubernetes Service (AKS). The Deployment resource creates three identical pods (replicas) running a container from a specified Azure Container Registry image, ensuring high availability and consistent application instances. The accompanying Service resource provides internal cluster networking by exposing these pods on port 80, using a ClusterIP type that allows internal communication within the Kubernetes cluster. Together, these resources enable a scalable and accessible containerised application deployment, with built-in load balancing and replication to enhance reliability and performance.

• Scaling Mechanisms

# Container Apps handles scaling automatically
scale:
  minReplicas: 0
  maxReplicas: 10
  rules:
    - name: http-rule
      http:
        metadata:
          concurrentRequests: "100"

The configuration defines scaling rules. It establishes a flexible scaling strategy that allows the application to automatically scale from 0 to 10 replicas based on HTTP traffic. The specific rule named "http-rule" sets a scaling threshold of 100 concurrent requests, meaning when the number of simultaneous requests to the application reaches or exceeds 100, the system will automatically increase the number of replicas to handle the load, up to a maximum of 10 instances. Conversely, during periods of low traffic, the application can scale down to zero replicas, which helps optimise resource utilisation and reduce costs by only running instances when needed.

#AKS requires explicit configuration
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: my-cool-app-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: my-cool-app
  minReplicas: 1
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 50

This Kubernetes Horizontal Pod Autoscaler (HPA) configuration automatically scales the "my-cool-app" deployment based on CPU utilisation. It targets the specified deployment and allows the number of pod replicas to dynamically adjust between 1 and 10 instances. The scaling rule is triggered when the average CPU usage across all pods exceeds 50%, meaning if the deployment's pods collectively consume more than half of their allocated CPU resources, Kubernetes will automatically add more replicas to distribute the load, up to a maximum of 10 pods. Conversely, if CPU usage drops, the number of replicas will be reduced, ensuring efficient resource utilisation and maintaining application performance under varying workload conditions.

The code snippets above are intended to provide an idea of the differences when it comes to deploying your containers to either environment and not provide a full working deployment. For AKS, getting to the container deployment stage requires a lot of plumbing to ensure your cluster is production ready and able to handle the workload. Moreover, there will be ongoing admin overheads to maintain the cluster. Given the above, the question of when to use what still remains.

When to Use What?

The Verdict

Is Azure Kubernetes dead? Far from it. While Azure Container Apps offers an excellent solution for many modern application scenarios, AKS continues to serve as the backbone for complex, enterprise-grade container orchestration. The choice between the two depends on your specific requirements, team expertise, and application architecture.

Think of Container Apps as a high-level abstraction perfect for teams wanting to focus purely on application logic, while AKS provides the full power and flexibility of Kubernetes for teams requiring complete control over their container infrastructure.

Rather than viewing them as competitors, consider them complementary services in Azure's container ecosystem. Many organisations successfully use both: Container Apps for simpler, event-driven workloads and AKS for complex, stateful applications requiring fine-grained control.

The future of containerisation in Azure likely involves both services evolving to serve their distinct use cases better, providing developers with the right tool for the right job. To the many engineers I have directed towards Container Apps, I hope this article summarises the various discussions we have had and can be used in future when met with the same question.

Benefits of Peer Code Reviews:

Early Detection of Issues:

One of the primary benefits of peer code reviews is the early detection of issues. By having multiple developers scrutinise the code, errors, bugs, and potential issues can be identified at an early stage. This proactive approach significantly reduces the likelihood of bugs reaching production, thereby minimising costs associated with bug fixing and maintenance.

Knowledge Sharing and Learning:

Peer code reviews naturally foster a culture of knowledge sharing and continuous learning among team members. During the review process, developers are exposed to different coding styles, techniques, and best practices. Reviewers can share their expertise, insights, and alternative approaches, enabling others to learn and grow. This collaborative environment promotes the exchange of ideas and encourages developers to stay updated with the latest industry trends and practices.

Improved Code Readability:

Peer reviews emphasize code readability and maintainability. Reviewers have the opportunity to suggest improvements to enhance code structure, naming conventions, and documentation. By striving for clean and understandable code, the overall readability and maintainability of the codebase are improved. This, in turn, leads to easier debugging, refactoring, and long-term maintenance.

Consistency and Adherence to Standards:

Peer reviews play a vital role in enforcing coding standards and ensuring consistency across the codebase. Reviewers can identify deviations from established guidelines and suggest necessary modifications to align with coding standards. This practice leads to a more uniform and cohesive codebase, making it easier for developers to understand and collaborate on the project. Consistency in coding style and practices also contributes to better code integration and reduces potential conflicts during code merging.

Enhanced Collaboration and Team Building:

Peer reviews encourage collaboration and foster a sense of camaraderie within the development team. Developers come together to review each other's code, exchange ideas, discuss implementation approaches, and share their insights. This collaborative environment strengthens teamwork, improves communication, and builds trust among team members. It also fosters innovation and creative problem-solving as different perspectives and expertise are brought to the table.

Tips for Effective Peer Code Reviews:

Define Clear Objectives:

To ensure effective peer code reviews, it is crucial to establish clear objectives and goals. Clearly communicate the purpose of the code review process, specifying the aspects to focus on, such as functionality, performance, security, or readability. This helps reviewers understand the specific areas they need to scrutinise, allowing for more targeted feedback and suggestions.

Encourage Constructive Feedback:

Creating a positive and respectful environment for the code review process is essential. Encourage reviewers to provide specific and actionable feedback, focusing on the code itself rather than criticizing the developer. Constructive criticism helps identify areas for improvement without demoralizing the developer. Encourage reviewers to provide not only feedback on issues but also highlight the positive aspects of the code.

Keep Reviews Focused and Manageable:

To ensure efficient and effective code reviews, it is essential to limit the scope of each review to a manageable size. Reviewing smaller chunks of code improves efficiency, allows reviewers to provide more detailed feedback, and makes the process less overwhelming. Avoid overwhelming reviewers with excessively large code submissions that may result in important issues being overlooked.

Use Tools for Efficiency:

Leverage code review tools that provide a structured framework for conducting reviews. These tools allow reviewers to leave comments directly on the code, track changes, and facilitate discussions. Popular code review tools include GitHub Pull Requests, GitLab Merge Requests, and Atlassian Crucible. Utilizing these tools streamlines the review process, ensures transparency, and provides a central platform for collaboration and documentation.

Rotate Reviewers:

To encourage diverse perspectives and prevent blind spots, it is beneficial to rotate the responsibility of code reviewing among team members. Different reviewers bring fresh insights and experiences, which can lead to the identification of issues that may have been missed otherwise. Rotation also helps distribute the workload more evenly, prevents reviewer fatigue, and ensures that everyone benefits from the learning experience of being a reviewer.

Follow Up on Feedback:

To maximize the impact of peer code reviews, it is crucial to encourage developers to actively address the feedback received during the process. Engage in meaningful discussions with reviewers, clarify any doubts, and resolve any concerns raised. Regularly revisit previous feedback to track progress and ensure that suggested improvements are implemented. This iterative approach fosters continuous improvement and helps developers grow their skills and knowledge.

Conclusion:

Peer code reviews offer significant benefits in improving code quality throughout the software development lifecycle. By leveraging the expertise and knowledge of team members, code reviews promote collaboration, enhance code readability, and detect issues early on. Implementing effective peer review practices in software development workflows leads to more robust, maintainable, and high-quality codebases. Embrace peer reviews as an integral part of your development process, and watch your code quality soar to new heights. The combined effort and collaboration of team members will ultimately result in software that is more stable, efficient, and aligned with coding standards. It is important, however, to be mindful that code reviews is just one tool for improved code quality and it should be used in combination with other tools.

In this article, I will explore the concept of minimal API in C# and look at how a similar approach is done in node js.

What is Minimal API in C#?

Minimal API is a way of building APIs that are designed to be lightweight and easy to use. Minimal API uses a feature introduced in C# 9 called top-level statements, which allows developers to write code without the need for a class or method.

Minimal API is based on the concept of convention over configuration, which means that developers can write less code by following a set of conventions. This approach reduces the need for boilerplate code, and it helps developers to focus on writing the core functionality of their APIs.

It also provides a simplified routing syntax that allows mapping endpoints to methods without the need for attributes or controllers. This approach makes it easy to build APIs without having to worry about the details of the routing system.

How to use Minimal API?

To use Minimal API, you need to install at least .NET 6 and create a new project. You can do this using the following steps:

1. Install .NET 6: To install .NET 6, go to the official .NET website and download the latest version of the .NET SDK.

2. Create a new project: Once you have installed .NET 6, open the command prompt or terminal and run the following command:

   dotnet new web -o minapi

   This command creates a new web project named "minapi" in the current directory.

3. Add a minimal API endpoint: Now, open the Program.cs file in the minapi project and replace the code with the following:

WebApplicationBuilder builder = WebApplication.CreateBuilder(args);
WebApplication app = builder.Build();

app.MapGet("/", () => "Hello World!");
app.Run("http://localhost:5000");

In this code, we are using the WebApplication.CreateBuilder method to create a new instance of the WebApplicationBuilder class. We call the Build method to create an instance of WebApplication Then we use the MapGet method to map a GET request to the root endpoint ("/") to a lambda expression that returns the string "Hello World!".

Finally, we use the Run method to start the web server and listen for incoming requests.

1. Run the application: To run the application, open the command prompt or terminal and navigate to the "minapi" directory. Then run the following command:

   dotnet run

   This command will start the web server and listen for incoming requests.

2. Test the endpoint: Now, open a web browser and navigate to http://localhost:5000/. You should see the message "Hello World!" displayed in the browser.

While minimal APIs have only been around since .Net 6, one would argue that this is driven by what has already been implemented in other languages and frameworks. Let's consider how a web server is set up in node js.

const http = require('http');

const server = http.createServer((req, res) => {
  res.statusCode = 200;
  res.setHeader('Content-Type', 'text/plain');
  res.end('Hello World\n');
});

server.listen(3000, '127.0.0.1', () => {
  console.log(`Server running at http://127.0.0.1:3000/`);
});

This server creates a basic HTTP server using the built-in Node.js http module. It listens on port 3000 of the localhost IP address (127.0.0.1). When a client makes a request to the server, the server responds with a simple Hello World message.

To run this server, save the code in a file with a .js extension (e.g. server.js), navigate to the file's directory in your terminal or command prompt, and run the command node server.js. This will start the server and print Server running at http://127.0.0.1:3000/ to the console. You can then open a web browser and visit http://localhost:3000/ to see the Hello World message. Note that you can only run this code if you have node installed on your machine.

A lot of developers have always liked how simple creating a server in node js is. With only a few lines of code, you have your node js server up and running. In C#, it has required a lot of boilerplate code and loads of pipeline configuration. Minimal API has changed all this!

Conclusion

Minimal API is a feature introduced in .NET 6 that simplifies the process of building APIs. It is designed to help developers write less code, reduce boilerplate code, and increase productivity. It provides a lightweight, streamlined syntax for creating APIs that are easy to understand, maintain, and extend.

In this article, I explored the concept of minimal API and looked at some examples to understand how it works. I also discussed how to use minimal API by creating a new project and adding a minimal API endpoint. C# developers now have a node js equivalent in terms of the simplicity of setting up a web service.

Records were introduced in C# 9.0 and provide a way to define immutable classes with value semantics. They are similar to classes but offer several advantages over traditional class definitions. Records can be reference types (class records) or value types (struct records).

Records are defined using the "record" keyword, followed by the name of the record and an optional set of properties enclosed in parentheses. For example, the following code defines a record for a person with a name and age property:

public record Person(string Name, int Age);

This record definition creates a new class record with two properties, Name and Age. These properties are automatically initialized when a new instance of the record is created, and cannot be modified after initialization. The above declaration is equivalent to below:

public record Person
{
    public required string Name { get; init; }
    public required int Age { get; init; }
};

Just like classes, you can define records with mutable properties by defining the set method.

Records offer several advantages over traditional class definitions:

1. Simplified syntax: Records provide a more concise syntax for defining classes, making it easier to create and maintain your code. This is achieved by generating a lot of boilerplate code automatically, reducing the amount of code that you need to write. Here are some examples of how C# records simplify syntax:

   1. Constructor: With traditional C# classes, you need to define a constructor for each property, which can be time-consuming and verbose. With records, a constructor is automatically generated, which means that you don't need to define it yourself. This simplifies the syntax and reduces the amount of code you need to write.

   2. Properties: In a traditional C# class, you need to define properties for each field, which can also be time-consuming and verbose. With records, you can define properties using a simplified syntax, like this:

       public string Name { get; init; }
      

      This syntax defines a read-only property called Name that can be set during object initialization.

   3. Equality: C# records generate an equality method automatically, which means that you don't need to write code to compare object values. The generated equality method compares all properties of the record for equality, simplifying the syntax and reducing the amount of code you need to write. Remember that two classes are equal if they have the same reference in memory. To achieve equality based on individual properties of two classes with different references, you need to override the Equals method and implement IEquatable interface.

   4. Built-in formatting for display: C# records generate a ToString() method that provides a default string representation of the record. This is useful for debugging and testing purposes, and it simplifies the syntax by eliminating the need to write a custom ToString() methods.

Overall, C# records simplify syntax by generating a lot of boilerplate code automatically. This reduces the amount of code you need to write, making your code easier to read, write, and maintain.

1. Immutable by default: In C#, immutability refers to the property of an object whose state cannot be changed after it has been initialized. In other words, once an object is created, its values cannot be modified.

   Immutability is important because it helps to prevent bugs that can occur when objects are modified unexpectedly. For example, if an object is modified by one part of your code, but not updated properly in another part of your code, you can end up with unexpected behaviour and hard-to-debug errors.

   In C#, there are several ways to create immutable objects. One way is to use the readonly keyword to declare fields that can only be set once, either in the constructor or in the field declaration itself. Another way is to use immutable collections, which are collections that cannot be modified after they are created.

   In addition to preventing bugs, immutability can also improve the performance of your code by allowing the compiler to optimize your code more effectively.

   Immutable objects can be safely shared between threads without the need for locking or synchronization, which can also help to improve performance.

   Overall, immutability is an important concept in C# and in programming in general. By designing your code with immutability in mind, you can create more robust, maintainable, and performant applications.

   Records are immutable by default, which means that their values cannot be modified once they are created. This makes them safer to use in multi-threaded environments and reduces the risk of bugs caused by unintended modifications.

2. Value semantics: Records are designed to have value semantics, which means that they are compared based on their property values rather than their memory addresses. This makes it easier to compare and work with records in your code.

   Value semantics in C# refers to the way that values are compared and passed around in memory. When a value type is used, its value is copied and passed around in memory. This means that when two variables of a value type are compared, they are compared based on their values, rather than their memory locations.

   For example, consider the following code:

    int a = 5;
    int b = 5;
    bool areEqual = a == b; // true
   

   In this case, the values of a and b are compared, and they are considered equal because they have the same value.

   Value semantics are different from reference semantics, which are used with reference types. When a reference type is used, a reference to the object is passed around in memory, rather than the actual object itself. This means that when two variables of a reference type are compared, they are compared based on their memory locations, rather than their values.

   For example, consider the following code:

    public class PersonClass
    {
        public string FirstName { get; }
        public string LastName { get; }
        public PersonClass(string firstName,string lastName)
        {
            FirstName = firstName;
            LastName = lastName;
        }    
    }
   
    PersonClass p1= new PersonClass("John", "Doe");
    PersonClass p2= new PersonClass("John", "Doe");
    bool same = p1==p2; //false
   

   In this case, the values of p1 and p2 are compared, but they are not considered equal because they have different memory locations.

   Value semantics are important in C# because they help to ensure that values are compared and passed around correctly in memory. This can help to prevent bugs and improve the overall performance of your code.

3. Pattern matching: Records can be used with pattern matching, which allows you to write more expressive and concise code. For example, you can use pattern matching to check if a record has a certain value or property.

To use records in your C# code, you first need to define a record using the record keyword. Once you have defined a record, you can create new instances of the record using the same syntax as a regular class:

var person = new Person("John Doe", 30);

You can access the properties of a record using the dot notation:

Console.WriteLine(person.Name); // Output: John Doe
Console.WriteLine(person.Age); // Output: 30

Records can be used with other C# features, such as inheritance, interfaces, and generics. For example, you can define an interface that requires a record with certain properties:

interface IPerson
{
    string Name { get; }
    int Age { get; }
}

You can then use this interface to create a method that accepts any record that implements the IPerson interface:

public void PrintPerson(IPerson person)
{
    Console.WriteLine($"Name: {person.Name}, Age: {person.Age}");
}

In summary, records are a powerful feature in C# that provide a more efficient and organized way to work with structured data. They offer several advantages over traditional class definitions including simplified syntax, immutability by default, value semantics, and pattern matching.

By using records in your C# code, you can create more expressive and concise code that is easier to maintain and less prone to bugs. Whether you are working on a small project or a large-scale application, records can help you improve the design and functionality of your code.

There are two main approaches to writing unit tests; code-first and test-first. In code-first, a developer writes the functional component of the application before writing unit tests for that component. This approach is popular with inexperienced developers as well as legacy code that has no unit tests in place. In both cases, writing unit tests as an afterthought is a difficult task. This is because it usually results in rewriting (refactoring) some of the code under test.

The test-first approach is where unit tests are written before actual functional code. Inexperienced developers struggle with this approach as it requires different thinking; writing unit tests for code that does not exist. With the test-first approach, no code needs refactoring in order to be testable. Unit tests written in advance also provide guardrails for functional code. It forces a developer to write code that is loosely coupled, maintainable and modular. It is, however, not a substitute for well-thought-out design patterns and clean code principles.

In my view, the benefits of unit testing cannot be overstated. There are arguments against unit testing mostly centred around resource management: time spent writing unit tests can be spent writing functional code. However, the opposite is true: unit testing saves you a ton of time in the long run! Some of the benefits of unit testing include:

1. Reduced bugs: One of the primary benefits of unit testing is the ability to detect and fix bugs early in the development process. By testing individual units of code in isolation, developers can identify issues before they are integrated into the larger codebase, making them easier to resolve. Imagine refactoring code that has no unit tests. For code with any meaningful complexity, It is hard in this scenario to be sure that refactoring has not introduced any bugs. With unit tests in place, you only have to make sure that the tests pass before and after refactoring.

2. Better code quality: Unit testing forces developers to write better code by ensuring that each unit of code not only behaves as expected but is also structured in a testable way. This leads to higher-quality code that is easier to maintain, modify, and extend. As pointed out earlier, a unit test acts as a guardrail for your functional code not just functionally but structurally as well. It will force the developer to write smaller, single-purpose methods and use abstractions rather than relying on implementations.

3. Faster development: Unit testing helps speed up the development process by identifying bugs early and preventing them from becoming more significant issues later on. This allows developers to focus on writing new code and improving existing code rather than fixing bugs. Unit tests also mean that less time is spent on manually retesting existing code when new code is introduced.

4. Simplified debugging: When unit tests fail, developers can easily pinpoint the location of the bug and fix it quickly. This helps simplify the debugging process and makes it easier to maintain the codebase.

Unit tests are not a silver bullet to bad coding practices or eliminating bugs in code bases. They are an effective tool when properly employed with the right mindset. When writing unit tests becomes about code coverage numbers, then they are not being employed effectively. Let's look at some of the downsides of unit tests:

1. Over-reliance on unit tests: It's easy to fall into the trap of relying solely on unit tests to catch bugs, leading to a false sense of security. Unit tests can only test individual units of code, and bugs can still exist in the larger codebase that is not caught by unit tests. This is why integration tests must be part of a test suit for any business application.

2. Time-consuming: Writing unit tests can be time-consuming, especially for large codebases. Developers must strike a balance between writing enough tests to ensure adequate code coverage while still delivering code on time. This is especially true in legacy code bases. In greenfield applications, it is easier to maintain good test coverage with less effort compared to legacy applications.

3. False positives and negatives: Sometimes, unit tests can fail for reasons other than bugs, such as environmental issues or test setup problems. These false negatives can lead to wasted time and effort and make it harder to identify real bugs. Sufficient effort needs to be spent to remediate the cause of false negatives and ensure that tests are reliable and consistent.

I want to end this article with an example of a unit test using the test-first approach. This example is trivial compared to what you will encounter in practice. However, it demonstrates the thought process when writing a unit test before actual code.

Let's say you are asked to implement a class that calculates the sum of two numbers. The first question will be what name makes sense for this class? I will call it Calculator. The second question is what methods will the class have? Since we have only been given one functional requirement, i.e. calculate the sum of two numbers, I will have a single method called Add. I expect this method to take 2 parameters i.e. the numbers to be added and return the result. There are additional questions you must ask yourself here such as what is the number type (int, double, decimal etc.), how will parameters be passed, what is the return type etc. Below is an example of how we can write a test for this imaginary class and method using the NUnit testing framework:

[TestFixture]
public class CalculatorTests
{
    [Test]
    public void TestAddition()
    {
        Calculator calculator = new Calculator();
        long result = calculator.Add(2, 3);
        Assert.AreEqual(result, 5);
    }
}

In this example, we're creating a new instance of the Calculator (class under test) class and calling its Add method with two arguments. We're then using the Assert.AreEqual method to check that the result of the Add method is equal to the expected value of 5. We do not care how Add is performing the actual addition of the two numbers. All we care about is that given inputs of 2 and 3, we should get 5 as the output. We can then go ahead and create our Calculator class with the Add method. When our unit test starts passing then we know our work is done.

In conclusion, unit testing is a critical aspect of software development that helps catch bugs early and improve code quality. In the long term, it speeds up development by enabling developers to only focus on new functionality and not manually retesting existing code. However, developers should be aware of the pitfalls of unit testing, such as false positives/negatives and over-reliance on unit tests, to ensure that they are getting the most out of their testing efforts.

Resilient systems are those that are designed and developed to withstand and rapidly recover from disruptions, failures, or unexpected events while maintaining continuity of operations and minimizing damages. Such systems are characterized by their ability to adapt to changing conditions, anticipate potential risks, and leverage redundancy and diverse resources to mitigate the impacts of disruptions. Resilient systems can be found in various domains, such as critical infrastructure, supply chains, healthcare, and cybersecurity, and are essential to ensure the sustainability and stability of operations in the face of uncertainty and volatility.

Resilient systems play a crucial role in ensuring the sustainability and stability of operations in the face of disruptions and uncertainties. They provide organizations with the ability to rapidly adapt to changing conditions, anticipate and mitigate risks, and maintain continuity of critical operations. Building resilient systems can lead to reduced downtime, losses, and reputational damage in the event of disruptions. Additionally, resilient systems can improve stakeholder confidence, enhance operational efficiency, and support long-term growth and competitiveness. In short, investing in resilient systems is vital for organizations that seek to thrive in today's volatile and interconnected environment.

Resilient systems should have the following elements:

• Availability: Availability refers to the ability of a system to remain operational and accessible to users. The people, processes, technology, data, and facilities all contribute to ensuring availability. Effective management of these elements can help ensure that the system remains available to users even during disruptions.

• Scalability: Scalability refers to the ability of a system to handle increasing workloads and data volumes. The technology and facilities elements are critical to ensuring scalability. The technology must be designed to scale up or down quickly to handle changes in workload, while the facilities must have adequate resources to support the increased workload.

• Maintainability: Maintainability refers to the ease and speed with which a system can be repaired and restored to normal operation after a disruption. The processes, technology, and facilities elements all play a critical role in maintainability. Effective processes can reduce repair times, while resilient technology and facilities can help ensure that the system can be restored quickly.

• Recoverability: Recoverability refers to the ability of a system to recover from disruption and return to normal operation. The people, processes, technology, data, and facilities elements are all critical to ensuring recoverability. Effective management of these elements can help ensure that the system can recover quickly and minimise the impact of disruptions.

• Security: Security refers to the protection of a system from unauthorized access, data breaches, and other malicious activities. The people, processes, technology, data, and facilities elements are all critical to ensuring security. Effective management of these elements can help ensure that the system remains secure even during disruptions or cyber-attacks.

Several design patterns can be used to ensure a system has some or all of the above elements of resilience:

• Circuit Breaker Pattern: The Circuit Breaker pattern is a design pattern that can help prevent cascading failures in a distributed system. In a distributed system, failures in one component can cause failures in other components, leading to a cascade of failures that can bring down the entire system. The Circuit Breaker pattern works by monitoring requests to a particular service, and if the number of failures exceeds a certain threshold, it breaks the circuit and stops sending requests to that service. This approach can help isolate failures and prevent them from spreading to other parts of the system, improving overall resilience. The Circuit Breaker pattern is often used in conjunction with other techniques such as load balancing and auto-scaling to improve system availability and scalability.

• Load Balancing and Auto-Scaling: Load balancing and auto-scaling are techniques used to distribute workloads across multiple instances and automatically adjust resources in response to changes in demand. Load balancing involves distributing incoming traffic across multiple instances to ensure that no single instance is overloaded, while auto-scaling involves automatically adding or removing instances in response to changes in demand. By distributing workloads across multiple instances, organizations can reduce the risk of overloading any one instance, and by auto-scaling, organizations can ensure that resources are always available to handle peak demand. This approach can help maintain system availability and scalability, even during periods of high demand.

• Backup and Disaster Recovery Strategies: Backup and disaster recovery strategies involve regularly backing up data and storing it in a safe location, and developing plans to quickly recover from a disaster. By regularly backing up data, organizations can minimize the risk of data loss, and by developing disaster recovery plans, organizations can quickly restore critical systems in the event of a disaster. Backup and disaster recovery strategies are critical components of any resilient system, as they help maintain system recoverability and availability.

• Redundancy and Fault Tolerance: Creating redundant systems is one of the most important best practices for designing resilient systems. Redundancy can involve deploying multiple instances of critical components, using distributed databases, or implementing failover mechanisms that can automatically switch to backup systems. This approach ensures that even if one or more components fail, the system can continue to operate without any disruption.

  In addition to redundancy, fault tolerance is also crucial for resilient system design. Fault tolerance refers to the ability of a system to continue operating even if one or more components experience failure. To increase fault tolerance, organizations can use techniques such as isolation, encapsulation, and graceful degradation. Isolation involves separating different components of a system so that a failure in one component does not affect other components. Encapsulation involves shielding components from each other to prevent failures from spreading. Graceful degradation involves designing components so that they can continue to function at a reduced capacity even if other components fail.

• Monitoring and Logging: Monitoring and logging are essential for identifying and diagnosing problems. There are various tools and services that enable organizations to monitor and log various aspects of their systems, including system health, performance, and security. By monitoring and logging key metrics, organizations can identify potential problems early and take corrective action before they become more serious. Monitoring and logging solutions can also provide real-time visibility into system performance, making it easier to detect and respond to issues as they arise. This approach can help improve system maintainability and recoverability.

By incorporating these specific techniques into the system design process, organizations can further improve the resilience of their systems. Each of these techniques addresses a specific aspect of resilience, such as availability, scalability, recoverability, and maintainability, and can help organisations develop systems that are better able to withstand disruptions and maintain continuity of critical operations. Overall, a comprehensive approach to designing resilient systems should include a combination of these techniques, tailored to the specific needs and requirements of the organization and its applications.

Specific practices are required for designing resilient systems. Moreover, even well-designed resilient systems require processes and practices to be in place to work effectively. Some of these processes and practices include:

• Automated Testing and Deployment: Automated testing and deployment can help organizations identify and address potential problems before they become more serious. By automating testing and deployment processes, organizations can catch issues early in the development cycle and quickly make changes to address them. This can help reduce the risk of outages caused by software defects or configuration issues.

  Automated testing and deployment processes can include tools such as continuous integration and continuous delivery (CI/CD). CI/CD allows organizations to automatically test and deploy code changes to production environments. By automating these processes, organizations can quickly identify and address issues, reducing the risk of outages and minimizing downtime.

• Incident Response Planning: Incident response planning is essential for quickly and effectively responding to system disruptions. By developing a comprehensive incident response plan, organizations can minimize the impact of system disruptions and quickly restore critical systems.

  An incident response plan should include clear procedures for identifying, diagnosing, and responding to incidents. The plan should also identify the roles and responsibilities of each team member involved in the response effort. Additionally, the plan should include communication procedures to ensure that all team members are aware of the incident and can quickly respond.

• Communication and Collaboration: Communication and collaboration are critical for effective incident response and maintaining resilient systems. By fostering a culture of open communication and collaboration, organizations can ensure that team members can quickly share information, identify potential problems, and work together to address issues.

  To improve communication and collaboration, organizations can use tools such as incident management systems and performance monitoring dashboards. Incident management systems allow teams to track incidents and collaborate on incident response efforts. Performance monitoring dashboards provide real-time visibility into system performance, allowing teams to quickly identify and address potential issues.

Overall, following these best practices can help organizations design and maintain resilient systems that can withstand disruptions and maintain continuity of critical operations. By creating redundancy, increasing fault tolerance, automating processes, planning for incidents, and improving communication and collaboration, organizations can build systems that are better able to adapt to changing conditions and provide reliable and consistent services to users.

Consider a simple example where we have a class called User and another class called Authenticator. The Authenticator class is responsible for authenticating users and relies heavily on the User class to do so. Here is an example of how these two classes might be implemented in a tightly coupled way:

public class User
{
    public string Username { get; set; }
    public string Password { get; set; }
}

public class Authenticator
{
    private User _user;

    public Authenticator()
    {
        _user = new User { Username = "admin", Password = "password" };
    }

    public bool Authenticate(string username, string password)
    {
        if (username == _user.Username && password == _user.Password)
        {
            return true;
        }

        return false;
    }
}

In this example, the Authenticator class creates an instance of the User class and uses it to authenticate users. This creates tight coupling between the two classes because the Authenticator class is dependent on the User class. Let's try to test this code. A test might look like below code:

public class Tests
    {
        [Test]
        public void WHenPasswordDoesNotMatchResultIsFalse()
        {
            Authenticator auth = new();
            bool authResult = auth.Authenticate("admin", "password1");
            authResult.Should().Be(false);
        }
    }

Any problems with the above test? The test passes fine but what happens when another developer updates the Authenticator class to create a user whose password is now password1? The test above will need updating to match the change in Authenticator class. Let's say we had a test for a positive case of a username and a password that matches, and negative cases for either username or password not matching. That's 3 tests already where you need to change the code.

Let's improve this a bit to avoid such changes.

public class User
{
    public string Username { get; set; }
    public string Password { get; set; }
}

public class Authenticator
{
    private User _user;

    public Authenticator(User user)
    {
        _user = user;
    }

    public bool Authenticate(string username, string password)
    {
        if (username == _user.Username && password == _user.Password)
        {
            return true;
        }
        return false;
    }
}

This is better! Our tests now do not rely on a user object created in the Authenticator class. We can create user objects and just inject them into Authenticator. However, Authenticator is still dependent on the User class. For each User to be authenticated, we need a new instance of Authenticator.

To fully decouple Authenticator and User classes, and avoid creating a new instance of Authenticator each time a user needs authenticating, we can use another object that is responsible for creating a User object. In fact, in a real application, users will be stored somewhere where a lookup can be performed when there is a login attempt. Let's look at the below code snippet.

public class User
{
    public string Username { get; set; }
    public string Password { get; set; }
}

public interface IUserRepository
{
    User GetUser(string username);
}

public class Authenticator
{
    private readonly IUserRepository _userRepository;

    public Authenticator(IUserRepository userRepository)
    {
        _userRepository = userRepository;
    }

    public bool Authenticate(string username, string password)
    {
        User user = _userRepository.GetUser(username);

        if (user != null && password == user.Password)
        {
            return true;
        }

        return false;
    }
}

In this example, we define an interface called IUserRepository that specifies the methods for getting a User object. We then modify the Authenticator class to accept an instance of this interface via its constructor. This enables us to inject a different implementation of the IUserRepository interface, depending on our needs. As a start, we may have users in a database for the actual application but we may also have users in a file for our unit tests. We can then implement a FileUserRepository for our tests and a DbUserRepository for actual application code.

You may also argue that you may want to abstract User class by creating an IUser interface that can be implemented by classes representing different types of users.

Developers sometimes write tightly coupled code because it feels easier at the beginning. Loosely coupled code requires thinking and planning upfront. However, tightly coupled code quickly becomes difficult to main as a change in one place may result in changes in many other places.

One practice that forces developers to put in the effort to implement loosely coupled code is Test Driven Development. Following the principle of the path of least resistance, a developer can write unit tests upfront with much thought into the structure of the actual implementation. However, when actual code is developed, this is where effort will be required to align with the unit tests.

Let's briefly look at records in Pascal. Just like in C#, a record in Pascal was a way to create complex structured data types. Let's create a Person record:

type Person = record firstName : string; lastName : string; city : string; end;

var person1: Person;

We can then use the created instance to assign and read individual properties using dot notation similar to modern languages.

person1.firstName = 'john';
person1.lastName = 'doe';

writeln('First name is: ', person1.firstName)

You can also access properties using with keyword. You can compare Pascal records using @ operator (@person1 = @person2). This is equivalent to ReferenceEquals in C#. You can also override comparison operators to customise the equality check of objects. Pascal records are limited in that they do not support encapsulation and do not enforce immutability.

C# records, what are they and for what

A record is a reference type, similar to classes, that provides value-based equality out of the box. C# 10 goes further to add record structs - value type records. You can achieve value-based equality using classes as well but you will not get any help from the compiler. Let's look at some examples to make sense of this:

//define a record with 2 properties
public record class PersonRecord (string FirstName, string LastName);

//create two instances of person record
PersonRecord personRecord1 = new PersonRecord("John", "Doe");
PersonRecord personRecord = new PersonRecord("John", "Doe");

Console.WriteLine($"{personRecord == personRecord1}");
Console.WriteLine($"{personRecord.Equals(personRecord1)}");

//Outputs
True
True

First, note the definition of the record class - there is no explicit definition of FirstName and LastName properties. However, you can still refer to them just as you would for class-defined properties i.e. personRecord1.FirstName. Second, you might be wondering how come both statements for comparing the two instances output true. Let's look at how you define a similar normal class:

public class PersonClass
{
    public string FirstName { get; }
    public string LastName { get; }
    public PersonClass(string firstName,string lastName)
    {
        FirstName = firstName;
        LastName = lastName;
    }
}


PersonClass personClass = new PersonClass("John", "Doe");
PersonClass personClass1 = new PersonClass("John", "Doe");
Console.WriteLine($"{personClass == personClass1}");
Console.WriteLine($"{personClass.Equals(personClass1)}");

//Outputs 
False
False

Notice how much code you have to write to define a normal class of the 'equivalent' record class. Even after writing so much more code, you get different results when comparing two instances of the class. To get the same results when comparing two different instances of the same class as that of record instances, you need to implement IEquatable<T> interface as well as override ==, != operators and GetHashCode.

What is happening under the hood is that in addition to generating public properties on the record type based on the constructor parameters, the compiler also generates an implementation of IEquatable<T> interface and equality operator overrides.

Records provide many additional features. Among them is non-destructive properties defined through constructor positional parameters as in the example record definition above. A ToString() override also provides formatted output for record instances. Record types can only inherit from other record types. They cannot inherit class or struct types and vice versa.

In addition to a record class, you also have a record struct. What is the difference? A record struct is a value type as opposed to a reference type of record class. Remember that for reference types, you only pass a reference to your object and not the actual object. For a value type, you pass the actual object. The decision of whether to use a record class or record struct is largely dependent on how large your object can be. If it is small enough not to be an overhead copying it around and you want to utilise the features of a record, then a record struct will make sense otherwise a record class is ideal.

The record type is one piece of evidence of what Microsoft's C# and Visual Studio teams have been paying attention to in the last few years: developer productivity. This is a topic for another day though but it's worth just mentioning here. In all, there is little similarities between C# and Pascal records!

Before diving into Azure Resource Manager (ARM) templates and Terraform, let's briefly look into what ARM is and how important it is to the deployment and management of resources in Azure. ARM provides a consistent layer through which you create, update and delete resources in Azure. Through ARM, you can also manage security, access control, locks and tags of your resources.

Imagine you want to create a virtual machine in your subscription. When you submit a request for the VM to be created, Azure Resource Manager first performs authentication and authorisation to ensure that the request is coming from a valid user and that the user has the authorisation to create a VM in the given subscription. Once authentication and authorisation are complete, the request is passed to the VM service which in turn will create the requested VM. Updates or delete requests for the VM also follow the same path.

To submit a request to Azure Resource Manager, one can use Azure portal, REST API endpoints, Azure Command Line Interface (CLI) or Azure PowerShell. Because all requests go through ARM, you get a consistent result no matter what tool you use to submit your request. Microsoft has done a great job ensuring that Azure Portal, CLI, PowerShell and REST API offer the same capability.

A newcomer to Azure trying out a few things might find the portal more user-friendly and can quickly get results without a steep learning curve. However, as soon as one gets past the initial stages of introduction to Azure, the Portal's limitations are easily exposed. The portal, in itself, is not limited but it limits the user. Let's look at some scenarios where Azure Portal limits the user:

• Recreating resources: when you create a resource through the portal, you need to go through the same steps again if you want to create similar resources e.g. additional VMs.

• Reproducing environments: your pilot in a DEV subscription has worked like a charm and you want all the resources in that environment to be recreated in a Test environment. You will need to manually recreate each resource manually.

The above two scenarios sound very much like an on-prem environment setup; you buy servers for your dev environment, set them up and check everything works as expected. Repeat the steps for your test, UAT, PROD and DR. The appeal of the Cloud for most organisations is to reduce the Total Cost of Ownership (TCO). The ability to get resources at a click of a button (a resource with exact same specification all the time!) reduces the cost of setting up new hardware to match the existing one.

Infrastructure as Code (IaC)

Azure enables developers to describe their infrastructure in form of code. This code can then be deployed and redeployed with the same result. Developers are familiar with the DRY principle and IaC guarantees that DRY is adhered to. You can define your entire infrastructure (VMs, Storage, Service Bus, Azure Cache for Redis, Web Apps etc) and deploy it to your dev subscription. When ready to create that infrastructure in your Test environment, at the press of a button you will get the exact copy of your dev infrastructure in Test. Any serious infrastructure setup and maintenance will use continuous delivery (CD) for deploying resources similar to CI/CD for deploying software. Microsoft provides an Azure DevOps platform for creating and running your infrastructure deployment pipeline. Other tools out there can also be used including GitLab and GitHub.

ARM templates

To implement Infrastructure as Code, you use ARM templates. A template is a declarative language using JSON or Bicep syntax to define Infrastructure. There are plenty of examples of templates for different resources on the web. If you are trying out ARM templates for the first time, bear in mind that templates adhere to a defined schema and most examples on the web are based on older versions. If you are copying snippets from one example to another, ensure that they are from the same version of the schema you are using otherwise you may get errors.

A template can have five sections:

1. Parameters: they provide placeholders for values that you can pass at runtime and are not hard coded. For example, a VM name is a candidate for parameterisation otherwise every time you want to create a new VM you will need to change your template. You can parameterise as much or as little as you want. You may want to provision low-specification VMs in the dev/test environments and high-specification ones in prod. By parameterising the VM spec, you will not need to change your template when deploying to these two environments - you just pass different parameter values.

2. Variables: variables generate values at runtime usually by combining one or more parameters with other values. For example, if you wanted your VM name to include environment, you may create a variable that combines the name and environment parameters.

3. User-defined functions: you can use this section to combine multiple standard functions into a complicated function.

4. Resources: this is the section where you define all the resources you want to deploy.

5. Output: this section is used to display output at the end of template deployment.

At first, ARM templates could only be defined using JSON syntax. Microsoft later developed Bicep as an alternative and more concise language. I can see developers preferring Bicep to JSON as the former would feel natural to a developer. Let's look at an example to demonstrate how concise Bicep is compared to JSON syntax:

JSON

"parameters": {
    "vnetName": {
        "type": "string",
        "defaultValue": "myCoolVnet"
     }
 }

Bicep

param vnetName string = 'myCoolVnet'

The above snippets are both defining a string parameter and assigning a default value.

Terraform

Terraform is an open-source tool developed by HashiCorp. It uses HashiCorp Configuration Language (HCL) to declaratively define Infrastructure as code. Terraform takes the idea of having a predictable and repeated IaC further than ARM templates.

Imagine you are experimenting with Azure. You have your IaC scripted in ARM templates and then suddenly you want to experiment with Amazon Web Services (AWS). How do you redeploy what you have in Azure to AWS? Terraform is a multi-cloud IaC language that allows the deployment of the same infrastructure to different cloud providers.

Conclusion

Infrastructure as Code is a powerful concept. It enables organisations to deploy and maintain infrastructure predictably and consistently. Using IaC, organisations can deploy resources to different environments or regions with little overheads. Azure provides ARM templates that use JSON or Bicep syntax as a solution. HashiCorp provides Terraform as a cross-cloud IaC tool that supports most of the major Cloud platforms.

With no further ado, I will highlight the key features released under each major version of .Net Framework and .Net.

What is .Net and .Net Framework

Microsoft defines .Net and .Net Framework as follows:

• .NET Framework is the original implementation of .NET. It supports running websites, services, desktop apps, and more on Windows.

• .NET is a cross-platform implementation for running websites, services, and console apps on Windows, Linux, and macOS. .NET is open source on GitHub. .NET was previously called .NET Core.

Microsoft .Net Framework 1.0

• The original version of .Net Framework was released in 2002.

Microsoft .Net Framework 2.0

• This was a major upgrade to the previous version and brought the following features:

• Generics - the need to write once and use everywhere as well as improving performance by avoiding boxing and unboxing

• Debugger edit and continue feature that every developer has come to love

• 64-bit support, improvements to ASP.NET and ClickOnce deployment

Microsoft .Net Framework 3.0

• Windows CardSpace - Microsoft's attempt to address issues with password security and user identity

• Windows Communication Foundation - aimed client-server communication models

• Windows Workflow Foundation - ability to implement business processes using a visual workflow.

• Windows Presentation Foundation - for developing desktop applications using declarative language syntax called XAML

Microsoft .Net Framework 3.5

• The main features included expressions trees, HashSet collection, and LINQ

Microsoft .Net Framework 4.0

• Garbage Collection - improved performance by introducing background garbage collection replacing concurrent garbage collection

• Dynamic Language Runtime - enabled runtime type binding.

Other features included

• Monitor.Enter(Object, Boolean)

• System.TimeSpan

• String.IsNullOrWhiteSpace

• String.Concat

• StringBuilder.Clear

• BigInteger and Complex Numbers

• Tuples

• Stopwatch.Restart

• System.Lazy<T>

• System.Collections.Generic.SortedSet<T>

Microsoft .Net Framework 4.5

• Support for arrays larger than 2 gigabytes (GB) on 64-bit platforms

• Background just-in-time (JIT) compilation

• Improvements to:

• Windows Presentation Foundation (WPF)

• Windows Communication Foundation (WCF)

• Windows Workflow Foundation (WF)

Microsoft .Net Framework 4.5.1, 4.5.2, 4.6, 4.6.1, 4.6.2, 4.7, 4.7.1, 4.7.2, and 4.8

• These are in-place updates to .Net Framework 4.5. Version 4.5 is in itself an in-place upgrade for .Net Framework 4.0. What this means is that all these versions use the same runtime despite new versions of assemblies with new features. Application targeting a lower runtime version than what is installed on the target machine will run without needing recompilation. However, if an application targets a higher version of the framework than what is on the machine, it may run just fine if it does not reference any APIs that are not present in the old version of the Framework

Microsoft .Net Core 1.0

• Released in 2016 as an open-source project and the start of a new journey to bring Microsoft .Net to other platforms such as Linux and Mac. Support for this version ended in 2019

Microsoft .Net Core 2.0

• This version brought improvements around tooling e.g. implicit running of restore during project build, support for C# 7.1 and Visual Basic and support for .Net Standard 2.0

Microsoft .Net Core 3.0

• Added support for .Net Standard 2.1, C# 8.0, and introduced new types such System.Index.

Microsoft .Net 5.0

• Improvement to System.Text.Json and Relational pattern matching

• Introduced Records, top-level statements

Microsoft .Net 6.0

• Support for C# 10

• Minimal APIs

• Single page applications

• Unified platform across browser, cloud, desktop, IoT, and mobile apps

• Performance improvements

• Improvements to Blazor and Blazor WebAssembly (Wasm)