One of the hallmarks of "the edge" in computing is the array of sensors, controllers, and microcontroller unit (MCU) class devices that produce data and perform actions. For Kubernetes to be a versatile edge computing solution, a cluster needs to easily find these leaf devices. Most of these devices, however, are too small to run Kubernetes themselves. How can they be leveraged by a Kubernetes workload? How can a Kubernetes Pod find or access their outputs? Akri is the answer!

Today, Microsoft is proud to share the Akri project, designed to electrify the edge for Kubernetes users. Akri is a new open source project that exposes these leaf devices as resources in a Kubernetes cluster. Akri, meaning "edge" in Greek, can also be read as an acronym, as Akri aims to be "A Kubernetes Resource Interface for the edge."

It provides an abstraction layersimilar to the Container Network Interface (CNI)but instead of abstracting the underlying network details, it removes the work of finding, utilizing, and monitoring the availability of leaf devices such as cameras, sensors, and so on.

Akri leverages and extends the Kubernetes device plugin framework, which was originally created to focus on advertising static resources such as GPUs and other system hardware. Akri takes this framework and applies it to the edge, where there is a diverse set of leaf devices with unique communication protocols and intermittent availability. Akri continually detects nodes that have access to leaf devices and schedules workloads for them. Simply put: you name it, Akri finds it and you use it.

Architecture and functionality

Akri is Kubernetes-native. Its architecture is made up of four key Kubernetes components: two custom resources (CRDs), a device plugin implementation, and a custom controller. The first custom resource, the Akri Configuration, is where "you name it," telling Akri the kind of leaf device you want to discover. Then, "Akri finds it," as the Akri Agenta Kubernetes device plugin framework implementationsearches for the leaf devices, checking for the availability of your desired ones. Once your device has been discovered, the Akri Controller helps "you use it." It sees each Akri Instance, which represents a leaf device, and deploys a "broker" pod that knows how to connect to the leaf device and utilize it.

Built to make the edge come alive

Akri is made for the edge, handling the dynamic appearance and disappearance of leaf devices. A user simply has to apply an Akri Configuration to a cluster, specifying the discovery protocolfor example, Open Network Video Interface Forum (ONVIF) protocoland the pod that should be deployed upon discovery, such as a video frame server.

Then, Akri does the rest. A user can also allow multiple nodes to utilize a leaf device, thereby providing high availability in the case where a node goes offline. Furthermore, Akri will automatically create a Kubernetes service for each type of leaf device (or Akri Configuration), removing the need for an application to track the state of pods or nodes.

Most importantly, Akri was built to be extensible. The project currently has ONVIF and udev discovery protocols, but more can be easily added by community members like you. The more protocols Akri can support, the wider an array of leaf devices Akri can discover.

Learn more and contribute

You can easily deploy and test Akri today on your edge cluster using K3s, MicroK8s, AKS-HCI, or any other certified Kubernetes distribution. To jump into using Akri, try our end-to-end demo, which discovers mock video devices and ultimately displays footage from the cameras on a streaming application. Want to learn more? Check out our documentation or hear from us at the Edge Conference on October 21, where we will be presenting this technology.

This project elevates the power of Kubernetes on the edge and is intended to reside in a community-governed foundation in the Kubernetes ecosystem. With your input and help, we can get there. Is there a device you want to use that Akri doesn't have a discovery protocol for? Join us on Slack, create an issue on GitHub, or implement the new discovery protocol with the Akri community. We're excited to continue to refine and extend Akri so that everyone can benefit from this project.

We are eager to see what you discover with this project and how it evolves. Together, we can build a more connected edge with Kubernetes.

The post Announcing Akri, an open source project for building a connected edge with Kubernetes appeared first on Microsoft Open Source Blog.

At first, I was bewildered by this comparison. It was as if people were suggesting that this year's Super Bowl would feature a showdown between FC Barcelona and the New York Yankees. But a few months into the new year, I am still being asked variations of the same question: "Who is going to win: Helm or operators?" This is my answer.

To start with, let's dive into the purpose of each technology. What problems does Helm solve? What about operators? From there, we'll look at the areas of overlap. In the end, we'll turn back to the question, and ask which technology "wins."

Helm: the package manager for Kubernetes

The very first version of Helm was released on Nov. 2, 2015. Kubernetes was at version 1.1.0 and the very first KubeCon was about to take place. But even in these early days, Helm proclaimed its vision:

Helm provides package management for Kubernetes

We published an architecture document that explained how Helm was like Homebrew for Kubernetes.

From the earliest days, Helm was intended to solve one big problem: How do we share reusable recipes for installing (and upgrading and uninstalling) things on Kubernetes? We looked at operating system package managers like Homebrew, Apt, RPM, and Chocolatey, and we saw parallels in Kubernetes. Even up through Helm 3, this has consistently been our vision.

Today, Helm has over a million downloads a month, and we are aware of dozens of Helm-based tools, and thousands of publicly available charts. The conclusion I draw from this is that Helm, as a package manager, has been a success.

Operators: a design pattern for operational knowledge

Almost exactly one year after Helm 0.0.1, Brandon Philips, then CTO of CoreOS, posted one of the most brilliant blog posts of the Cloud Native era: Introducing Operators: Putting Operational Knowledge into Software.

Philips pointed out that we often relied upon humans to manage the runtime needs of applications. But with a system like Kubernetes, much of the material once set down in run books and user manuals could now be transformed into code.

An Operator is an application-specific controller that extends the Kubernetes API to create, configure, and manage instances of complex stateful applications on behalf of a Kubernetes user. It builds upon the basic Kubernetes resource and controller concepts but includes domain or application-specific knowledge to automate common tasks.

CoreOS illustrated this new design pattern in an operator designed to manage an etcd cluster.

Operators contained the institutional knowledge necessary to manage some, if not all, of the operational aspects of a workload inside of Kubernetes. As Philips suggests:

An Operator builds upon the basic Kubernetes resource and controller concepts and adds a set of knowledge or configuration that allows the Operator to execute common application tasks. For example, when scaling an etcd cluster manually, a user has to perform a number of steps: create a DNS name for the new etcd member, launch the new etcd instance, and then use the etcd administrative tools (etcdctl member add) to tell the existing cluster about this new member. Instead with the etcd Operator a user can simply increase the etcd cluster size field by 1.

Philips is absolutely correct: SREs and DevOps engineers spend far too much time manually re-running sequences of commands. The operator design pattern provides a compelling solution: write per-application tooling that codifies common management tasks.

Over the last few years, the notion of an operator has matured, partly due to the completion of Custom Resource Definitions (CRDs) and partly due to the Kubernetes API maturing. These days, authors of operators are likely to talk about how they wrote CRDs and custom controllers to manage applications. While the terminology has changed slightly, Philips' vision is very much still at the heart of today's operators.

Today, there are more than 100 operators available at varying degrees of stability. Leaders in the cloud-native space, including RedHat, IBM, and Microsoft, have released operators. The operator pattern is clearly a successful part of the cloud-native ecosystem.

So, what's the problem?

Helm is a package manager for Kubernetes. Operators are design-pattern-driven pieces of code that encapsulate knowledge for running an application. Yet as I noted at the article's outset, there are questions floating around about which one is "the winner."

With two markedly different technologies, why are we attempting to pit them against one another? Is this not like asking which sports team is better, the New York Yankees or FC Barcelona? In many discussions with a variety of people, I began to understand why people have arrived at the conclusion that Helm and operators are competitors. There are two causes:

• The two technologies share a set of common terms, such as "installation," "management,” and "lifecycle."
• If stretched to its boundaries, each tool can be made to do some semblance of the other tool's job.

Same terms, different meaning

For starters, let's cover the terminology issue. A skim through the documentation for each project will turn up a number of common terms. For example, both will talk about installing or creating resources inside of a Kubernetes cluster. While the terms are the same, though, the meaning is subtly different.

When Helm users talk about installing a thing, we mainly mean something like this:

I want to find a chart and be able to pass in some configuration and have that chart installed into my cluster without ever having to edit Kubernetes YAML.

To that end, Helm has focused on a standard packaging format, a template language for parameterization, and a system designed to easily locate and install off-the-shelf packages.

When the user of an operator talks about installing, what they mean is more like this:

I want to create a resource or resources as Kubernetes YAML and have those things spin up and maintain an application for me.

Thus, if you look at the instructions for installing the etcd operator, you will see that you are given guidelines and examples for creating your own Kubernetes YAML file, which will (when installed into the cluster by another tool) create and maintain an etcd cluster.

When Helm users talk about "management," they are thinking of being able to use a simple tool to see what applications are running, which resources belong to whom, and then perhaps be able to upgrade, rollback, or delete individual applications.

In contrast, when it comes to operators, "management" is often used to talk about the "day two ops" of an application: Managing data integrity, scaling an application up or down, or automatically recovering from a failure.

We could go on to other terms, but the core message is clear: We have a limited (and frequently overloaded) vocabulary that sometimes causes us conceptual headaches. But when we talk it out, we realize that we're dealing with different ideas.

Push to the limits

While both Helm and the operator pattern have their sweet spots, we can definitely push our tools beyond their intended use cases. In fact, we might be able to push the Helm Chart metaphor into performing some operator-like tasks or write an operator that also does some of Helm's package management tasks for itself. In so doing, we might be able to better compare the two technologies. But this is somewhat like asking the soccer player and the baseball player to engage in a kicking contest to see who wins. One is definitely better prepared.

I have seen some truly remarkable Helm charts. One, in particular, was over one megabyte of YAML and could orchestrate hundreds of components configured in an innumerable variety of ways. The chart could not only be used to install things, but also to repair broken clusters and keep all of these systems in sync. (It also used a special chart installer to encapsulate some extra logic.) As amazed as I am by this chart, I do see it as pushing beyond the bounds of what Helm is designed to do.

I know of some operators as well that push up against Helm on the installation and upgrade story, with custom installers that execute in-cluster (sort of like CNAB packages) and provide a similar workflow to Helm's install/upgrade/delete story. Again, there is nothing wrong with this. But it stretches well beyond Brandon Philips' definition of an operator.

In both cases, though, we must ask ourselves: are we stretching our tools because it is the best overall strategy? Or are we stretching our tools because we have gotten dogmatic about those tools (or, perhaps, because we fear to learn more tools)? We are doing a disservice to the Kubernetes community when we become so entrenched in our chosen tool or tools that we start rewriting them to repel other tools.

Operators and Helm charts have been working together since the early days of these technologies. For example, there are currently around sixty operators installable via the Helm Hub. There are even a few Helm operators (notably WeaveWorks' Flux operator for Helm) in which Helm functionality is provided using an operator that links directly to the Helm APIs. These are clear indications that the two technologies can work well in concert without having to push each technology past its limits.

Do we have a winner?

One might be tempted to look at the numbers and draw conclusions. Helm has more charts, more available tooling, and more users. But this would lead to an incorrect conclusion. After all, Helm is designed to make it easy to write charts and distribute them. Operators, in contrast, are tremendously difficult to write because by design they encapsulate complex operational knowledge. While the typical Helm chart is a few hundred lines of YAML, the typical operator is thousands of lines of code. Thus we would expect the technology to evolve slowly, with emphasis on covering the most widely used services.

On the contrary, one might be tempted to look for a win based on a notion of "architectural purity." On more than one occasion, I have heard fans of operators tout them as superior to Helm charts because they are based on CRDs and controllers (two central architectural features of Kubernetes). Helm is, in this estimation, "just templated YAML." This argument also isn't decisive. It ignores whether a problem is solved and begins with an argument by fiat that the solution to the problem must use controllers and CRDs. The problem Helm tries to solve is not better solved with CRDs and controllers (though there are available tools that use controllers to install Helm charts). Neither CRDs nor operators are necessary pieces for telling the package management story. Adding them simply increases the complexity (and the attack surface) of a package management system.

Again, it feels like we're back to the question of which sports team is best, which leads us to a concluding analogy.

In the early 2000s, artist Howard Schatz published a book of photographs of the top athletes from a multitude of sports. From gymnastics to basketball to sumo wrestling, Schatz posed them standing side-by-side. Each of these athletes had honed their bodies to achieve prowess in their chosen sport. Yet the contrast in size and shape could not have been more pronounced. Could we have looked at those athletes, picked one, and said, "you are the best athlete"? Not in any meaningful way. The best we could do is say "you are the best in your sport."

Operators have a different focus than Helm. Each has strengths and weaknesses. And there is definitely some overlap between the two. But we should be unsurprised to see the two tools work well together. After all, they are complementary rather than opposed.

Collectively, we would be better served by forgoing the mentality of dueling technologies. Instead, we should focus on using the combination of these technologies to truly make Kubernetes an easier platform to operate.

The post Is there a Helm and Operators showdown? appeared first on Microsoft Open Source Blog.

In the spirit of innovation, I’m thrilled to announce our new open source effort to enable trusted execution environments for Kubernetes. Trusted execution environments or “enclaves” are a hardware-backed secure execution environment that can ensure processes and their memory are secure while they execute. Today, we’re enabling trusted computing on Kubernetes anywhere via the Open Enclave SDK.

We’re also releasing a resource plugin that makes Encrypted Page Cache RAM a resource that the Kubernetes scheduler can use to make scheduling decisions. The number of enclaves on a CPU is limited, and this plugin ensures that Pods that need enclaves will be guaranteed to land on a node with an enclave available. This scheduler support is critical to running trusted compute environments in cloud-native applications via Pods.

Beyond these innovations for secure computing, I’m incredibly proud of the work that the Helm community has done to build and release Helm 3.0 last week. The vast majority of workloads deployed to Kubernetes are deployed via Helm, and Helm 3 is the next step in this journey. Over the past few years, the Helm team has carefully listened to user feedback about what was working and where changes were needed.

Of the many fixes and improvements, the most popular is probably the removal of Tiller from the cluster, making Charts more Kubernetes native and more secure by default. Speaking of security, the recent glowing independent security review of the Helm code base shows how dedicated and careful the Helm community has been in building a tool that is not just incredibly useful, but also secure as well. Many congratulations to the Helm community on this important milestone.

Just like the Helm team, in Azure, our open source work begins by listening to our customers. In particular, our customers in IoT and telecommunications. This feedback led us to understand how important it was for Kubernetes to support both IPv4 and IPv6 addresses for the same Pods in Kubernetes. Major kudos are due to Kal Henidak for his dedicated and tireless work in engineering both the code and design changes necessary to support multiple addresses per Pod. As you might imagine this change required careful work and coordination across the entire Kubernetes code base and community. Kal’s hard work in collaboration with the SIG-Networking community is being recognized with a shared keynote with Tim Hockin. Plan on attending the keynote to learn more about IPv4 and IPv6 in Kubernetes!

Finally, by combining both open source community and innovation we have a remarkable collection of open source projects reaching important milestones at KubeCon. The newly announced Buck (Brigade Universal Controller for Kubernetes) project shows how Cloud Native Application Bundles (CNAB) with Brigade radically simplify the development of new operators. The Kubernetes-based Event-driven Autoscaling (KEDA) has shown incredible community interest. It's a great collaboration between Azure Functions, Red Hat, and others. Here at KubeCon, the KEDA community is hitting the 1.0 milestone and is stable and ready for production use. I also want to congratulate the Cloud Events community on their recent 1.0 release and I’m excited that Azure Event Grid has correspondingly added support for the 1.0 version of Cloud Events. Cloud Events is a CNCF project for an open and portable API for event-driven programming and it’s awesome that it is available in a managed environment in Azure.

Of course, containers and DevOps are a year-round focus for my teams beyond KubeCon. We've been busy this fall.

In the four weeks since we launched the Distributed Application Runtime (Dapr) project, we have seen strong interest from the community and have been listening to the many stories of how people are using Dapr in their projects, including modernizing Java code, building games, and integrating with IoT solutions. The breadth across different industries is amazing to see. The interest in the Dapr runtime repo has grown beyond our expectations. It's been awesome to see the community come together and continue the momentum. We are excited to announce the release of Dapr v0.2.0, focusing on community-driven components, fixes across the Dapr runtime and CLI, updates to documentation, samples, and the addition of an end-to-end testing framework. You can find out more about the v0.2.0 release at the Dapr repo.

Just building distributed systems isn't enough, you need to be able to observe how they are running in production, and the CNCF Prometheus project has emerged as a de-facto standard for exposing metrics on all sorts of servers. But it’s still easier to integrate with cloud-based monitoring rather than run your own metrics server. To enable this, Azure Monitor for containers can scrape the metrics exposed from Prometheus end-points so you can quickly gather failure rates, response per secs, and latency. From Log Analytics, you can easily run a Kusto Query Language (KQL) query and create your custom dashboard in the Azure portal dashboard. For many customers using Grafana to support their dashboard requirements, you can visualize the container and Prometheus metrics in a Grafana dashboard. Azure monitoring combines the best of open technology with the reliability of a cloud service.

In the last few years, KubeCon has grown from a single-track to many tracks and thousands of people. For me personally, and the community in general, it’s been an incredible journey. I’m excited to see people in San Diego please stop by the Azure booth and say hello!

The post Empowering cloud-native developers on Kubernetes anywhere appeared first on Microsoft Open Source Blog.

Helm is the well-known and much-used package manager for Kubernetes. If Helm is new to you, you're in luck! Join in using Helm, one of the most popular tools in the cloud native toolbox it's great for managing the complexity in your applications, sharing configurations, and easily versioning your updates for maintainability.

Helm 3 (now available!) is an evolutionary improvement, bringing enterprise-grade security and increased usability. Enhancements in Helm's security and stability are in direct response to and in coordination with community needs, as Helm is a widely-used Incubating Cloud Native Computing Foundation (CNCF) project, with an open source community numbering in the hundreds of contributors shaping its decisions.

Being almost as old as Kubernetes itself, Helm originally evolved with some configuration options which were parallel but not entirely congruent with Kubernetes. Rethinking Helm 3 from the ground up let the community bring Helm's permissions model, command-line switches, RBAC, and more in line with current Kubernetes implementations. The Helm team edited the Helm 3 architecture carefully and removed the server-side component known as Tiller, which was obviated by improvements to Kubernetes in the years since Helm 2's design. Helm 3 is simultaneously simpler and supports more modern security, identity, and authorization features of Kubernetes.

"Sounds exciting!" you may say. "How do I get started?" You can use Helm 3 immediately whether or not you've used Helm before. If you have Helm charts you've been using with Helm 2, you will want to read about essential changes, but most charts will work unmodified. A migration guide and Helm 2to3 plugin will help you make the move. And you may find that while you don't need to rewrite your charts right away, you'll delight in the new library charts, which will help you de-duplicate configs you use across many charts for consistency and security.

Release information is now stored in the namespace with the release. This enables you to use a release name on a per-namespace basis, instead of being limited by all releases needing to share the same Tiller namespace. (One side effect of the removal of Tiller!) And the re-imagined three-way strategic merge patches allow the old and new state on disk to be examined in the context of the live state in the running cluster. This prevents unexpected incidents caused by uncommitted production updates.

“But wait…” comes the worried exclamation. "I don't have time to change anything right now!" Good news: Helm 2 will get bug fixes backported for six months and security patches for a year, so you can update at the pace that fits your organization's needs. And backward compatibility is emphasized if you want to keep using previous command-line flags, in many cases that option is available. The Helm FAQ covers the breaking changes, and the Helm blog post on this major version release dives into the technical details you'll want to know.

Of course, any major shift in an open source project comes with changes, and the upstream chart repository of yore will be one of those we're looking at pushing charts to Open Container Initiative (OCI) registries to ameliorate limitations of the Chart Repository API. Experimental features like OCI support are a great place to look if you'd like to get involved with the direction of the project testing and feedback are valuable contributions.

Open source means giving back to the community and the Helm Go SDK has proven so handy for our needs that we've refactored it to allow for broader use. If you've integrated Helm into other projects, we'd love to hear from you!

Whether in person at KubeCon + CloudNativeCon next week or on our weekly community calls, Slack, mailing lists, GitHub, and more, the Helm maintainers are delighted to hear what's working, what needs improvement, and how you're using Helm to make package management in your Kubernetes clusters simpler and more secure.

The post Helm 3: Simpler and more secure appeared first on Microsoft Open Source Blog.

Why Helm?

Many teams already rely on Helm 2 to deploy and manage their applications on Kubernetes, the open source project that has become the de facto open source distributed systems kernel. Kubernetes usage is reportedly above 70% in large organizations as of 2018. At Microsoft, we see customer uptake of the managed Azure Kubernetes Service growing rapidly, and a great deal of our industry is focused on this space.

Kubernetes orchestrates containers, typically as a collection of services that together enable a microservice application in which various services work together to provide a larger experience. To host these workloads, many different Kubernetes components must be configured. In addition, Kubernetes has no built-in concept of an application as a logical, manageable unit, which makes application operations more difficult unless an organization dedicates staff to focusing on those primitives.

Typically, we prefer to make it simpler for cluster users to deploy and manage their applications as logical units in a self-service fashion. That's where Helm adds value!

Helm is the package manager for Kubernetes applications

Last year, the CNCF's cloud native survey made it clear that the "preferred method for packaging is Helm (68%) followed by managed Kubernetes offerings (19%)." Users find that Helm is a great way to:

• Manage complexity: describe complex Kubernetes applications in a "chart."
• Share charts: search for shared charts on public and private chart repositories.
• Easily update Kubernetes applications: in-place upgrades and rollbacks (which are actually roll-forwards; Helm doesn't include time travel!) to past versions, using the release history of charts.

Making complex things easier to manage is the hallmark of a good tool and the strong adoption of Helm shows us that many people are looking for this kind of tooling.

Helm 3 is built for production scenarios in mind

Adventure. Excitement. An on-call engineer craves none of these things operational surprises don't help us sleep at night. Predictable and repeatable production-ready software we can operate at scale is delightful and that's why we create tools like Helm.

If you're using Helm already, try Helm 3 today to help the community ensure there are no surprises for your use cases. Helm 3 is the result of years of community contributions and conversations that clearly show how organizations are using Helm and how they need it to evolve for their production use cases.

Even if Helm is new for you, your input is welcome! Take a look at Helm 3 and find out how Helm charts help you bring operational simplicity and enterprise-ready stability to your Kubernetes environments.

Simpler to use, more secure to operate

Wasn't Helm 2 already simple? If we install the same Helm chart with Helm 2 and Helm 3, the application installed will be precisely the same, just as we would expect! So, what's the motivation behind Helm 3?

Tiller, the server-side component of Helm 2, requires additional security steps and Helm 2 was created for developers to install applications when Kubernetes did not yet have role-based access control (RBAC). This complexity isn't needed in recent releases of Kubernetes, so for Helm 3 we were able to remove tiller entirely.

As a result, Helm 3 is significantly simpler, while also supporting all the modern security, identity, and authorization features of today's Kubernetes. Helm 3 allows us to revisit and simplify Helm's architecture, due to the growing maturity of Kubernetes identity and security features, like role-based access control (RBAC), and advanced features, such as custom resource definitions (CRDs).

Join the cloud-native fun!

We're so excited for what Helm 3 will allow you to do! You can read about all the new features, such as chart reuse in chart libraries, in the blog series Helm 3: Charting Our Future and join the discussion on GitHub to make this major release the best Helm for you. We're looking forward to connecting during sessions and deep-dives at KubeCon EU this week and can't wait to hear your feedback on the alpha version.

Your feedback is invaluable, as the Helm community intends to make Helm 3 generally available (GA) at Helm Summit EU in September. Come share your stories and continue the conversation!

The post Helm 3: simpler to use, more secure to operate appeared first on Microsoft Open Source Blog.

These conferences are fantastic because they represent an opportunity to meet with our users and learn about how our software is both making them more successful and also hindering their success. While learning about how we empower developers is awesome, it's even more important (and significantly more humbling) to learn about the areas where they have unmet needs, have hit bugs, or are otherwise blocked by choices we've made or bugs that we've written.

Listening to our users and customers and focusing our energy on solving their real-world problems is the #1 (and #2 and #3) focus for my teams. In that spirit, I'm excited to announce: the Helm 3 release, Kubernetes integration with Visual Studio Code (VS Code), the Virtual Kubelet project 1.0 release, and the Service Mesh Interface (SMI), which is our new community project for collaboration around Service Mesh infrastructure.

Helm 3

The first, and arguably most significant for the Kubernetes community, is the first Alpha release of Helm 3. Helm 3 represents a nearly complete re-factoring of the Helm package manager to evolve it from its origins to a modern application package manager. The Helm project is nearly as old as Kubernetes itself. As a result, its original design pre-dated many advancements in Kubernetes like CustomResourceDefinitions and even Kubernetes RBAC.

Because of this, the Helm 2 architecture was forced to implement a number of features itself, which made it less tightly integrated with Kubernetes, and meant that managing things like RBAC of Charts and Resources was complicated and disconnected from Kubernetes itself. Helm 3 eliminates this impedance mismatch.

By replacing custom APIs for charts and deployments with CustomResourceDefinitions, things like Kubernetes RBAC directly apply to Helm and the whole system feels significantly more tightly-integrated and Kubernetes-native. You can now use the 'kubectl' command line to interact with your Helm charts and Kubernetes native RBAC to limit access and resources that users can create.

Helm has become the defacto standard for packaging and deploying Kubernetes applications by focusing on the end-user and enabling those users to be successful. The advances and improvements in Helm 3 continue this trend and make it even more useful to both existing users, as well as others who may have previously tried other solutions. Many congratulations to the Helm project and community on reaching this milestone!

Kubernetes extension for Visual Studio Code

Making Kubernetes accessible to everyone is a challenge that I think about every day. It's great that we've built a system that makes it easier to deploy distributed systems, but if we haven't made it possible for everyone to use Kubernetes then we've actually failed. In that vein I have spent a bunch of time recently working with the team that builds the open source Kubernetes extension for Visual Studio Code.

This extension brings native Kubernetes integration to VS Code. You can easily view the contents of your cluster, see the state of pods at a glance, right-click to get a terminal in a Pod or port-forward network traffic, and easily filter your way through logs to identify problems all within the exact same environment where your code lives.

Furthermore, in keeping with our open principles, the extension is open source on GitHub and works with Kubernetes anywhere. No matter where you are running Kubernetes, the VS Code integration makes it easier to work with your applications and clusters with fewer windows and context switches.

Today at KubeCon EU in Barcelona, I'm excited our VS Code integration has reached the 1.0 milestone and is fully supported for production management of your Kubernetes clusters. Even more importantly, we've also added an extensibility API that makes it possible for others, like Red Hat OpenShift, to build their own integration experiences on top of our baseline Kubernetes integration. ('yo dawg, I heard you like extensions, so I added extensibility to your extension.') Much like Custom Resources, adding extensibility enables collaboration at the core while enabling others to build rich experiences targeted to specific environments. It's a testament to the value of an open and extensible approach to tooling.

Virtual Kubelet 1.0

Speaking of community, I'm also incredibly excited to see the Virtual Kubelet hit the 1.0 milestone. The Virtual Kubelet represents a unique integration of Kubernetes and serverless container technologies, like Azure Container Instances. Enabling people to free themselves from the toil of managing an operating system, while still using Kubernetes for orchestration, is a powerful value proposition to startups and enterprises alike.

I'm really excited that this community is healthy and reaching this important milestone, and that Azure was able to play a role. Earlier this month at //build we also announced the general availability of AKS virtual nodes powered by the open source community code in the Virtual Kubelet project. Again, showing the value of open source even when delivering tightly integrated features for Azure.

Service Mesh Interface (SMI)

Finally, I'm thrilled to see the community that is developing around the Service Mesh Interface specification. It has been clear to us for a while that users and customers are excited by the promise of Service Mesh to provide the next advances in cloud-native application development. However, it is also clear that the monolithic integration of interface and implementation that previously existed for Service Mesh has limited their adoption.

The world of Service Mesh is fast and evolving and users are concerned that if they are locked to any particular implementation, they may be locked into a complex surface area. By providing a generic API interface that is then implemented by various service mesh providers, like Istio, Linkerd, and Consul Connect, the Service Mesh Interface frees users to use service mesh capabilities without being bound to any particular implementation.

This means that they are free to experiment, and even change implementations without having to change their applications. This brings Service Mesh on parity with other parts of Kubernetes, like Ingress, Container Runtime (CRI), and Networking (CNI), which all have generic interfaces with pluggable implementations. We've really enjoyed working with our great partners on the SMI specification and we look forward to building an even larger community as we move forward and iterate on the specification.

Enjoy KubeCon and Barcelona (and make sure to check out the Sagrada Familia). The Azure team is will be busy learning from you, our community, so we can build the next set of great products and features that make you even more productive and happy!

Questions or feedback? Let us know in the comments below.

The post Extending Kubernetes in the open appeared first on Microsoft Open Source Blog.

Build Spring Boot 2.0 apps with Azure Starters and new VSCode extensions: Spring is an open source application framework developed by Pivotal that provides a simplified, modular approach for creating Java applications. This week, the Azure team rolled out additional support for Spring, including Spring Boot Starters for Azure and new Java and Spring extensions for Visual Studio Code, where you can build production-ready apps and easily deploy them to the cloud. See the complete announcement on the Azure blog or check out the Azure Spring Boot Starters page on GitHub to get started.
Public preview of Java on App Service, built-in support for Tomcat and OpenJDK: This week, the Azure team announced the public preview of Java apps on App Service. This release includes built-in support for Apache Tomcat 8.5/9.0 and OpenJDK 8, making it easy for Java developers to deploy web or API apps to Azure. Bring your .jar or .war file to Azure App Service and Azure will take care of the capacity provisioning, server maintenance, and load balancing.

OSI's new ClearlyDefined project: ClearlyDefined is a brand new incubator project from the Open Source Initiative (OSI) that focuses on crowd-sourcing critical licensing and security data for open source projects. The project aims to increase clarity around open source projects, making it easier to build a community and gain contributors. Learn about how to get involved here.
SCaLE 16x: Southern California Linux Expo (SCaLE), the largest community-run open-source and free software conference in North America, expects to host 150 exhibitors this year, along with nearly 130 sessions, tutorials and special events. The team is proud to support the sixteenth annual event, SCaLE 16x. If you're attending the show, swing by our booth for a pro photo by @JulianCashPhoto and connect with experts in open source software with cloud app development, DevOps, serverless, containers, machine learning, AI and more. Learn more about our talks here.
New Open Source Show series: The new Open Source Show debuted this week, featuring Microsoft Cloud Developer Advocates, Lena Hall and Suz Hinton, as they discussed data ingestion, stream processing, and sentiment analysis pipeline. In their demos, you'll learn about setting up a data ingestion and processing system, using Twitter client, Event Hubs and Spark on Azure Databricks as an example. Lena and Suz also discuss alternative options for stream processing, and how it can be used for various scenarios, including IoT, and how to apply machine learning to streaming data by showing an example of sentiment analysis on tweets coming in real-time. You'll start seeing code around minute 2:40.

CosmosDB and .NET Core: Jeremy Likness discusses ComosDB and what makes it a unique and flexible NoSQL cloud offering. Learn about the CosmosDB support for multiple APIs, including table storage and MongoDB, all accessible from .NET and .NET Core applications.

Here is some of the documentation supporting the VS Code and Spring updates this week.
Python support in Visual Studio Code: The Python extension for VSCode works with different Python interpreters as well as Anaconda. It leverages all of VS Code’s power to provide auto complete and IntelliSense, linting, debugging, and unit testing, along with the ability to easily switch between Python environments, including virtual and conda environments. Lots of documentation here.
Spring on Azure: These quickstarts and tutorials will show you how to develop and deploy Spring apps to the cloud. Documentation includes Spring Boot Starters announced earlier this week.
If you like these weekly updates, follow us @OpenAtMicrosoft to stay in touch daily.

The post Open Source Weekly: VSCode Python updates, Helm Summit learnings, and more appeared first on Microsoft Open Source Blog.

Where we came from and the current focus of the Helm community

We kicked off Helm Summit with a history of the Helm project and how it got to where it is today. The rest of the morning was spent seeing how community members push Helm to its limits with CI/CD using Helm and managing thousands of releases with an operator like Lostrmos. Several speakers also showed their work with the popular new tool ChartMuseum and how they used it in testing and deployment.
The second half of the day contained some deep dives on deploying multiple environments with Helm and how to test large chart repositories. Bitnami and Microsoft detailed the latest iteration of KubeApps with its support for Open Service Brokers and one-click installs of applications. Finally, we heard about the hot topic of how to secure Tiller, what parts of Kubernetes Helm interacts with, and how to secure those endpoints.
We closed out the day with lightning talks from across the entire community spectrum. Rajashree Mandaogane showed us a handy trick for rolling deployments when a ConfigMap changes, while Jimmy Zelinskie and Nikhil Manchanda both proposed ideas for a public App Registry or Chart Repository for Helm users, much like how DockerHub and Quay is for users of Docker.
What stood out most to us was the diverse set of companies and use cases presented. All of these talks were enlightening for the community as we learned about the different ways in which people used Helm. These were a perfect way to set the stage for the needs of the community as we went into Day 2 talking about the future of Helm.

The Future of Helm

The second day was all about the future of Helm, and it was exciting to see so many people invested and looking forward to participating in the discussions. Matt Butcher and Matt Fisher (2 of the 3 core maintainer Matts) set the guidelines for the day on how to approach planning Helm 3 as a community.
Throughout the morning, several people from the community presented well thought out proposals for Helm 3. Others gave 1-2 pitches for topics they wanted to discuss on the future of Helm. These included:

• Unified aliases, dependency, and value mapping functionality
• Tiller implementation using CRDs and a custom controller
• Better chart searching
• Measuring Helm usage
• Improvements for managing charts

Brian Grant presented an interesting idea to separate Helm and Tiller into composable tools for the Kubernetes ecosystem. This is an idea that many people expressed interest in implementing.
We finished off the morning with a Q&A panel with the Helm and Charts core maintainers. What struck us as most interesting was the number of questions related to the management of charts and how to handle the growing number of them in the repository.
The rest of the conference was spent in unconference-style working groups, where we discussed the various proposals and ideas pitched by others in attendance. We used this time to gather requirements and debate different ideas and needs.

Requirements

So then, what are the requirements we found? The full notes from the working groups can be found on GitHub, but we have broken up these requirements into 3 categories below for Helm, Tiller, and Charts. These are what we have identified as suggestions for requirements from the Helm community .

Helm

• Users should be able to push/upload a chart to a chart repository using the Helm CLI
• There should be a way to specify release dependencies (e.g. a release named "my-release" must exist in order to deploy the release they are installing)
• Users should be able to dynamically override non-templated values in an object
• Template engine plugging should be easier
• Templating should use an overlay to add on additional data
• Helm linter should allow for flexibility and customization
• Helm should have full YAML support (for things like anchors)
• There should be a curated list of plugins
• Plugins should have a starter or library to use so people don't have to write it all from scratch
• Re-work the existing `helm test` framework so releases can be more thoroughly tested (this is a very large requirement, see the notes for more details)

Tiller

• Tiller will become a custom controller that leverages some sort of CRD
• Remove Tiller entirely in favour of a pure client-side implementation
• Tiller should have a way to interface with application lifecycle management (the current idea being to have a way for Operators to report back to Tiller on the status of the application lifecycle)
• Tiller must be able to enforce RBAC rules for a user making the request
• Support external storage engines for Tiller

Charts

• Backwards compatibility must be maintained with Charts (all changes will need to be a new chart API version)
• Better measurement of chart usage metrics
• More performant chart index (whether through a new v2 spec or a distributed index)
• Create a chart adoption process for bringing charts into the main charts repository

We will be discussing these requirements in upcoming dev calls to solidify the actual requirement and then we will move them to the Helm repository.

Open Questions

Based on the discussion, there are several open discussion points that need to be resolved.

• How much of Brian Grant's proposal to separate the parts of Tiller should be implemented? There is a good case from the Helm developer side of things that would make things more modular. However, from the user perspective, this may not be desired as it has the possibility to make things harder to use (more assembly required)
• Should template rendering be moved client-side again?

Thanks

From all of the Charts and Core maintainers, thank you so much for all of the support to hel(m) make this summit a success. The talks and energy at the conference were nothing short of spectacular, and we are truly proud to be a part of this community. We are looking forward to scheduling other Helm Summits in the future in Europe and in North America again so we can reach as much of the community as possible. The support from the Helm community is what made this event great and we look forward to continuing to work with all of you.
Any questions for feedback? Please let us know in the comments.

The post What we learned at the first Helm Summit appeared first on Microsoft Open Source Blog.