It has been an incredible journey bringing Dapr from its initial release in October 2019 to production readiness and it could not have been made possible without a passionate, open community of developers from around the world. This is the benefit of open sourcethe hundreds of contributors to Dapr prove that many hands make for light work. Having many developers use Dapr, building applications in different languages and in a wide spectrum of scenarios ensures that Dapr is used and tested thoroughly. This was an important part of delivering Dapr v1.0 and will remain important as Dapr moves forward.

Since the initial announcement of Dapr, we've seen a rapid, steady growth of developers contributing to the project in all areas, and today we have 700 individual contributors to the project. This includes developers from cloud vendors such as Alibaba Cloud, large companies such as HashiCorp, as well as enterprises such as ZEISS who is building business critical systems with Dapr in production. Additionally, smaller startups such as Roadwork have made contributions while using Dapr to gain portability for their cloud native solution and accelerate its development for more agility.

A great example of this kind of contribution is the recent addition of the Dapr PHP SDK. This SDK was contributed by a developer from Automattic who is both passionate about PHP and excited about the possibilities Dapr holds. Contributions go beyond code and come in the form of docs, samples, issues, and guidance to Dapr users like the one provided by New Relic on how Dapr observability can be used with their monitoring tools.

As illustrated above, contributors have been making an impact in all corners of the Dapr project. Noticeably, the areas that have seen contributions from the greatest number of community members are the core areas of Daprthe Dapr runtime, which includes the code of the Dapr sidecar, the docs, and the components-contrib, the repository holding the code for components.

Having many contributors build new components is especially valuable, as components offer Dapr users the ability to leverage the Dapr API and interface with a wide range of technologies and systems. Having a large number of components available means helping even more developers simplify and accelerate the process of writing distributed applications. More components also means more flexibility in where these applications can be deployed, making code even more portable in multi-cloud or hybrid scenarios.

Today Dapr offers over 70 different components for state management (such as Redis, MySQL, Azure Cosmos DB, and many more), Pub/Sub messaging (such as RabbitMQ, AWS SNS/SQS, Kafka and more), secret stores (such as Kubernetes secret store, Azure Key Vault, HahiCorp Vault, and more) and both input and output bindings to integrate with services such as SendGrid, Azure Events Hub, and more.

Building an inclusive, open community has always been a high priority for the Dapr project. Last September, we shared the open governance model Dapr follows. We are committed to keeping Dapr vendor-neutral and welcoming to all developers and community members. To this end, we have begun evaluating open-source software foundations for Dapr's home in the future.

We invite you to try Dapr yourself and join the community. We believe that Dapr can help you build distributed applications more easily and we are always looking for more developers to provide feedback, open issues, and help us define the future of Dapr.

Learn more about Dapr

• Get started with Dapr
• Explore the Dapr documentation
• Learn more about Dapr v1.0

Join the Dapr community

• Join the community on Discord
• Join the Dapr community calls and watch recordings of past calls
• Follow @daprdev on Twitter

The post The community effort that delivered Dapr v1.0 appeared first on Microsoft Open Source Blog.

It’s no surprise that open source adoption and usage grew significantly this year. New data from GitHub's 2020 Octoverse report shows there were over 60 million new repositories created this past year, and more than 56 million developers on GitHub. When people had to stay home, developers came together to find community and connections through open source. And though open source developers had a lot of established remote practices, this year challenged companies of all sizes to integrate their open source software experiences and development models in new ways, bringing new learnings as a result.

We wanted to share four places where Microsoft is learning from and growing our engagement in open source over the last year that we hope can be useful for any developer or team looking to build and collaborate in 2021.

1. Seeking different perspectives makes better software

Success in open source is just as much about your own contributions to the community as it is about what you learn from the community. Behind every pull request, issue, and code snippet, is a person. It's important to connect with themto listen, learn, and empathize with them. They offer a different perspective and feedback that your team may not be thinking of.

I hear conversations in meetings (one of the new virtual hallways) about making sure we get feedback from industry users who are well outside the Microsoft faithful. With this new feedback, I hear a collective sound of Microsoft's perspective expanding and our gratitude for the new and different views we are receiving.

One example of community feedback changing our perspective was when the Dapr project received a lot of user feedback requesting a streamlined API to retrieve application secrets. The Microsoft team working on Dapr had not planned that work in the current cycle, but the community made it very clear that this new API would solve a lot of problems that developers were facing.

The Dapr maintainers worked closely with community members who submitted multiple PRs to add this functionality, covering everything from code to documentation to samples. After this was added, we found that customers also picked up this functionality and used it in their Dapr implementation.

This reminded us that listening to community feedback is extremely valuable, and that given opportunity, encouragement and support, community members will contribute effort to make requirements a reality.

2. Finding the balance between policy and autonomy

To help drive Microsoft's open source efforts, we have an Open Source Programs Office (OSPO), whose goal is to help our employees consume and participate in open source safely, effectively, and easily.

Over the last year, we have heard from more and more enterprise customersfrom retailers to banks to auto makerswho are looking to establish similar offices and practices internally. We share and discuss best practices on how to find the balance between setting policy while also empowering employees to do the right thing. While OSPOs will look different depending on your company's needs, a few common practices we often discuss include creating a cross-functional group, setting clear policies (and making them easy to find and understand!), investing in tooling, and providing rewards and motivation. We've shared our guidance and policies and we look forward to continuing to build out our own internal practices, and to share our learnings along the way to help others do the same.

3. Securing every link in your supply chain is critical

Using open source in your development process has many advantages, including increased time to market, reduced cost of ownership, and improved software quality. However, open source, like any software, has its risksopen source can contain security defects that lead to vulnerabilitiesand new research shows security vulnerabilities often go undetected for more than four years before being disclosed. Because open source software is inherently community-driven, there is no central or single authority responsible for quality and maintenance. Source code can be copied and cloned, leading to outsized complexity with versioning and dependencies. Worse yet, attackers can become maintainers and introduce malware.

As more systems and critical infrastructure increasingly rely on open source software, it's more important than ever that we build better security through a community-driven process. Securing open source is an essential part of securing the supply chain for every company. In 2020, we came together alongside GitHub, Google, IBM and others to create the Open Source Security Foundation (OpenSSF). The group is helping developers with resources to identify security threats to open source projects, providing education and learning resources, and finding ways to speed up vulnerability disclosures. In the coming year, the OpenSSF looks to provide hands-on help to improve the security of the world's most critical open source projects.

4. Over communicate

Big companies and big open source projects know that important information has to be communicated broadly and frequently across different channels. Even with this knowledge, Microsoft had to change rapidly this year just as so many other companies did. We no longer had moments of serendipitous interaction where you learn something helpful from bumping into someone in the coffee line, walking with a colleague to a meeting, or waiting with someone for the elevator.

This year, we learned the importance of over communication, which has been a hallmark of open source communities. Over communication is key because uncertainty can be more stressful than either good or bad news.

Take, for example, the Kubernetes projectit has never had an office and today they have 407 chat channels, which run the gamut from regional user groups to developer discussions about particular technology subsystems. These chat roomswhether they are IRC channels, Twitter hashtags, Teams, or Slack *are* the offices of open source projects.

While chat rooms are the new water cooler, they are temporal and transient. They are not the new announcement email or documentation repository. In the same way that no one is expected to know what happened in every meeting or conversation in the office kitchen, few people read the history of chat rooms when they return to their desk. Understanding how communication has changed and what expectations are set for every medium allows internal communication to remain a critical support of a good collaborative culture.

Looking ahead to 2021, together

These four investment areas are just as important to good corporate culture and health, as they are part of open source collaboration. We strongly believe that most of the hard (and, by that we mean interesting) problems of today will take a team or the whole industry to solve. This means we all need to be trustworthy and (corporately) self-aware participants in open source.

A few years ago if you wanted to get several large tech companies together to align on a software initiative, establish open standards, or agree on a policy, it would often require several months of negotiation, meetings, debate, back and forth with lawyers… and did we mention the lawyers? Open source has completely changed this: it has become an industry-accepted model for cross-company collaboration. When we see a new trend or issue emerging that we know would be better to work on together to solve, we come together in a matter of weeks, with established models we can use to guide our efforts.

As a result, companies are working together more frequently, and the amount of cross-industry work we're able to accomplish is accelerating. In 2020 alone, Microsoft participated in dozens of industry groups, associations, and initiativesfrom long-standing established organizations, like the Linux Foundation and Apache Foundation, to new emerging communities like Rust and WebAssembly. This work across companies and industries will continue in the year ahead and we look forward to learning, growing, and earning our place in open source.

Check out more in our 2021 developer insights blog series:

• Software development in 2021 and beyond by Amanda Silver
• Low-code trends: Why low-code will be big In your 2021 tech strategy by Dona Sarkar
• Looking back on software development in 2020 and forward to 2021 by Scott Hanselman

The post 4 open source lessons for 2021 appeared first on Microsoft Open Source Blog.

Dapr integration in the Azure API Management (APIM) service is now available. This new capability enables operations teams to directly expose Dapr microservices as APIs and make those APIs discoverable and easily consumable by developers with proper controls across multiple Dapr deploymentswhether in the cloud, on-premises, or on the edge.

Since its initial release last year, Dapr has been helping developers build stateless and stateful applications with any language or framework. By codifying the common microservice patterns, like service discovery and invocation with build-in retry logic, publish-and-subscribe with at-least-once delivery semantics, or pluggable binding resources to ease service composition using external APIs, Dapr is quickly becoming the de facto runtime for modern, distributed applications.

The larger number of Dapr deployments across multiple environments also comes with the increased complexity of API management. Using the Dapr integration in Azure API Management announced today, users are now able to apply the self-hosted gateway feature of APIM to manage all of their Dapr APIs, along with other APIs, in a single interface. This allows developers to decouple APIs from their backend service lifecycle. The more than 50 built-in policy templates can quickly accelerate the API management with things like TLS connection termination, payload formation, throttling enforcement, and caching across their deployments via an always up-to-date portal.

Using API Management with Dapr APIs

To illustrate the Dapr integration in APIM, we are going to walk through the process of securely and reliably exposing a public API that allows users to invoke a specific method on a single service deployed in Dapr. This is a common service invocation scenario where users may have any number of Dapr services, but only a small subset of them need to be available externally.

APIM supports two other Dapr building blocks, pub/sub, and resource bindings. See our reproducible demo at the end of this page for an in-depth walk-through for each one of these use-cases.

API Configuration

To define an API you can use any of the formats supported by APIM. In the example below, we are using the OpenAPI specification format.

You can import this file into APIM using either the Azure CLI shown below, or the APIM service in Azure portal:

API Operation Policy

To integrate this API with our Dapr microservice we will use a simple policy. The APIM policies encapsulate common API management functions and are composed into a series of steps that are sequentially executed on each request. These policies can be defined inside of inbound, outbound, and backend blocks. To configure the "/echo" operation invocation to be forwarded to Dapr API, we will define out a policy within the inbound section.

The "<base />" element at the beginning in-lines all the policies defined higher up in the chain at the API or even APIM service level (e.g., throttling or authorization). Placing that element before all others ensures that APIM evaluates it before forwarding the invocation to the back-end service.

Routing the inbound request to Dapr is as simple as setting the backend service using "backend-id" attribute to "dapr" that tells APIM to forward the invocation in its original format to the Dapr API implemented by the Dapr sidecar container running alongside self-hosted gateway. The "dapr-app-id" and "dapr-method" attributes allow us to set the Dapr application ID and method name where we want this API invocation to be forwarded.

The entire service discovery and actual invocation, including possible protocol translation between HTTP and gRPC, retries, distributed tracing, and error handling, are all done by Dapr. And since the external mapping of the API user invocations to Dapr is done in APIM policy, it can be easily and safely remapped to any other version as the API implementation evolves over time.

As that service evolves and potentially has additional versions, which may require different payload formats, APIM can manage the entire process through either its revisions, for non-breaking changes, or versions, in cases where API consumer opt-in is required.

Self-hosted Gateway deployment

To enable APIM to manage the APIs exposed by your Dapr services you will need to deploy and configure the self-hosted gateway into your Kubernetes cluster using these steps. To enable Dapr support in the gateway, you first augment the Kubernetes deployment template with a few Dapr annotations:

These annotations tell the Dapr control plane to inject a Dapr sidecar into the APIM gateway pod, which gives it the ability to invoke the Dapr API. To learn more about the Dapr Kubernetes sidecar configuration see Dapr docs.

The self-hosted gateway by default deploys two replicas to ensure availability during upgrades. Once ready, each replica will now have a second container (daprd) in the pod.

With the APIM gateway configured, you can invoke the "echo" service using the cluster ingress, while giving the operators the ability to manage their Dapr APIs using a single interface.

Give it a try!

This post only covers the service invocation capability of the Dapr integration with APIM. You can find a full demo repo with step-by-step instructions on how to configure this and other Dapr APIs integrations in APIM on GitHub. The complete reference of APIM Dapr integration policies can be found on the Microsoft Azure site. We are excited to see what you can build with this Dapr and APIM gateway integration. You can reach us either in the Dapr community Gitter channel or in our community calls.

The post Announcing Dapr integration in Azure API Management Service appeared first on Microsoft Open Source Blog.

Currently, this extension is focused on being used with the Azure Functions runtime hosted in Kubernetes or IoT Edge and is not currently available in the Azure Functions hosting plans.

Late last year, Microsoft announced Dapr, a distributed application runtime. Dapr provides a set of cloud-native building blocks including state management, service invocation, publish and subscribe, distributed tracing, and more. Dapr runs as a sidecar to any application to provide these capabilities across cloud and edge, integrating with any programming model or framework, including Express, Flask, ASP.NET, or Functions.

Azure Functions, in addition to the serverless offering within the Azure cloud, provides an event-driven runtime and programming model for those apps. Azure Functions enables you to write event-driven code that can trigger and scale on events, with a number of out of the box triggers and bindings to push your data. It's a purpose-built runtime for event-driven applications, providing capabilities and tooling to run these apps anywhere.

Azure Functions and Dapr working together

This new extension allows you to more easily pair the Azure Functions programming model with capabilities from Dapr. For example, you may wish to keep some persistent and flexible state alongside your function. You can now create and retrieve state managed by Dapr directly within the functions programming model like below:

This state example would work alongside a Dapr persistent state store, deployed as a Dapr component, which can be interchanged for any of the many Dapr state stores.

Publish and subscribe between Azure Functions

As another example, you can use the Dapr extension to publish and subscribe to topics between functions. Extending on Dapr state, the code below shows an example of using Dapr publish and subscribe with a JavaScript Azure Function. The Azure Function will publish data to a topic named myTopic.

Another function can now subscribe to this published topic and trigger whenever data is sent.

Dapr extension bindings and triggers

These are a couple of examples of how to use the Dapr extension. This extension has support for triggering functions on a Dapr service invocation, pub/sub events, or Dapr input/output bindings. It provides integration to use Dapr state, secrets, topics, and bindings from directly in your function code. The Dapr SDKs and contracts are managed completely through the Azure Functions host, enabling developers to surface these capabilities quickly and efficiently. Below is the list of bindings and triggers available.

Give this a try!

Today the extension is supported in any environment that supports running Dapr and Azure Functions hostprimarily self-hosted and Kubernetes modes. Azure Functions, Dapr, and this extension are all fully open source and can be run across cloud, hybrid, and edge. You can get started with this extension on GitHub, and follow this quickstart to build a JavaScript function that uses Dapr for state management and pub/sub across functions in a Kubernetes cluster.

We'd love for you to try out this Functions extension for Dapr, give feedback by raising issues in the repo and become a member of the community.

Questions or feedback about this release? Let us know in the comments below.

The post Announcing Azure Functions extension for Dapr appeared first on Microsoft Open Source Blog.

On top of the community response, we have also been busy working with customers around the world and from different industries to use Dapr for developing business-critical workloads. These customer engagements have been a tremendous source of feedback and participation, helping shape the roadmap for Dapr and putting Dapr on a path towards a stable v1.0 release. In future blog posts, we'll share some of these real-world scenarios where Dapr is helping companies build their future applications.

What is Dapr?

Dapr is an event-driven, portable runtime for building microservices on cloud and edge and embraces the diversity of languages and developer frameworks. Dapr codifies best practices for building distributed applications and at its core are API abstractions that provide common capabilities needed to build microservices, including state management, service invocation, pub/sub messaging, secrets management, and others.

Each capability is abstracted through a "Building Block" that defines a simple and consistent API decoupled from the technology chosen to implement it through the use of pluggable components. This best practice of decoupling frees developers from the burden of integrating and having a deep understanding of the underlying technologies they use. The use of building blocks also allows swapping implementations when the application is deployed in different hosting environments or when needs change as the application evolves without affecting your code. Developers are free to use all, a few, or even just one building block as needed.

Since Dapr itself provides both HTTP REST or gRPC APIs, applications using Dapr are portable and can be written in any language. Furthermore, Dapr runs as a sidecar to the application so a developer using Dapr simply sends a call to the Dapr sidecar and Dapr takes care of the rest. Since Dapr is a process or container that runs alongside the application it can run in any environment including a local dev machine (running Linux, Mac, or Windows), a Kubernetes cluster, an ARM edge device, or any other compute infrastructure. Dapr is also not limited to either on-premises or cloud environments it can run alongside any application, written in any language, deployed to any environment.

• To learn more about how Dapr read this overview.
• To try it yourself, see Getting started with Dapr.

Dapr evolution

Dapr has evolved considerably in the five versions released since October 2019. These are some of the highlights made in the past six months:

More building blocks, more components

Secrets building block – The first example of a community-driven addition to Dapr is the introduction of the Secrets building block which enables developers to retrieve secrets from secret stores that can then be used to authenticate to other resources, such as database access. The secrets building block takes away the complexity of authentication and secret retrieval with the store and turns this into a local API call. Currently supported secret stores include, HashiCorp Vault, Azure Key Vault, AWS Secret Manager, GCP Secret Manager, and Kubernetes.

More components – In addition, there have been many contributions of components, extending Dapr's support for commonly used technologies across all building blocks. For example, Dapr was launched with support for two state stores (Redis and Azure Cosmos DB) and since has added components for a total of 16 different state stores, including Memcached, MongoDB, AWS DynamoDB, among others. To explore more components, see the components contribution GitHub repo.

Extended developer tools support

SDKs – While Dapr is independent of any programming language, it can be easily integrated into applications with language-specific SDKs. As we see more developers using Dapr, our investment in these SDKs to help streamline Dapr integration has increased. Dapr offers SDKs for JavaScript, Python, Java, .NET, Go, as well as recently added Rust and C++. In addition, based on community feedback many improvements have been added into the Java SDK, included virtual actors, typed classes, and integration with the Spring Boot web framework. For .NET there have also been improvements to the types classes and integration with ASP.NET Core web framework. Similar support is planned for the other SDKs as part of the future road map.

Visual Studio Code extension – Being able to develop applications on your local machine without any cloud dependency is important for productivity and cost and is a key goal of Dapr. The Dapr Visual Studio Code (VS Code) preview extension helps developers debug applications using Dapr, interact with the Dapr runtime, and combines with the Dapr CLI. You can find the extension in the Visual Studio Marketplace.

Dapr dashboard This Dapr dashboard is a web UI that helps you visualize information about Dapr instances running on your local machine or on Kubernetes.

Increased security

Making sure that messages cannot be sniffed or tampered with in a distributed application is vital to prevent any malicious behavior. Dapr provides end-to-end secure encryption via mutual TLS. This includes encryption of network traffic between the sidecars and from every sidecar to the Dapr control plane. Mutual TLS is turned on by default for any application using Dapr and is configurable to rollover certificates on a frequent basis, for example, every hour. Users can bring their own certificates or use the built-in certificate authority Dapr provides to ease certificate generation and management. Learn more about Dapr security features.

Observability

As the Dapr sidecars run alongside services, it offers advanced observability capabilities including tracing, logging, and metrics collection. Dapr currently uses the Open Census standard and is moving to the Open Telemetry standard as it moves to a stable version. The adoption of standards, where possible, is a key goal of Dapr. For example, all pub/sub events also conform to the v1.0 Cloud Events specification

Tracing Tracing provides visibility of the calls between your services to create an application map and help diagnose issues in production. Dapr uses middleware to intercept traffic and adds correlation-IDs to enable distributed tracing using W3C Trace Context. Since Dapr integrates at the application level it enables asynchronous tracing on top of synchronous tracing meaning, for example, intercommunication via pub/sub can be traced. Furthermore, tracing is handled by the Dapr sidecar, so no code changes are required to instrument an application.

Metrics Metrics provide visibility into call latency times between services and resource usage. Dapr provides metric collection for both Dapr services (sidecar and the Dapr control plane system services), as well as the user application services. Metrics collected include CPU and memory usage, sidecar injection failures, request error rates, and many others. Dapr allows easy integrations with technologies such as Prometheus or Azure Application Insights so users can easily set up metric visualization. See how to setup Dapr with Prometheus and Grafana as an example.

What's next on the roadmap?

As Dapr matures, the focus of the upcoming work is towards the v1.0 release, getting Dapr out of preview and to production quality. This includes finalizing APIs, making stability improvements including long-running tests, extending end-to-end testing, performance testing infrastructure including releasing benchmark numbers, external security audits and threat model analysis, and taking contributions from the community.

To ensure that we are confident in getting to stable v1.0 release, we are working closely with several companies to take Dapr into production, deployed into a number of different environments, and with a diverse set of developer frameworks, including .NET, Go, Java, C++, Node.js, and others.

Join the Dapr community

We'd love for you to try out Dapr for yourself and become a member of the Dapr community. You can get involved in many ways, including:

• Explore the GitHub repos. Read the docs, comment on issues or open new ones, and contribute through pull requests. Look for issues labeled as "good first issue" to start with, for example, in the dapr/dapr or the dapr/components-contrib.
• Ask questions and join the discussions in the Dapr community Gitter channel, follow @daprdev on Twitter, or join the community calls held every two weeks (or view recordings of previous calls).

It's an exciting time to be a developer in the era of microservice development and we are excited to have you take part in the Dapr journey.

The post How Distributed Application Runtime (Dapr) has grown since its announcement appeared first on Microsoft Open Source Blog.