Programming with Palermo

Jeffrey Palermo, Microsoft MVP, Author, Speaker, Clear Measure Chief Architect, Azure DevOps Expert

Monthly Archives: October 2018

Jeffrey Palermo

Damian Brady on DevOps for Data Science and Machine Learning - Episode 008

This week, your host, Jeffrey Palermo, interviews Damian Brady. Damian is a Senior Cloud DevOps Developer Advocate at Microsoft, helping customers implement DevOps methods on the Microsoft platform. He's been with Microsoft for just over a year now and formerly served as a developer for Octopus Deploy.

In this episode, Damian and Jeffrey talk all things data science and machine learning. Damian answers key questions such as: what has been the biggest change in the area of data science since the Azure DevOps release? What does source control look like for data science projects in DevOps? And more. He also explains some of the interesting architectures he has put together for machine learning and walks Jeffrey through the process of his machine learning model from source control, building, packaging, and finally, to deploying. He also gives his recommendations for those who want to go even further with data science after listening to this week's episode.

Topics of Discussion:

[:52] About today's guest, Damian Brady.

[1:06] Damian introduces himself and explains his role at Microsoft.

[1:46] Which group Damian is presently on at Microsoft.

[4:14] With the Azure DevOps release, what's the big change in the area of data science? What is going to be different for people building or running models?

[6:47] For data science projects what does the source control look like?

[8:49] For the Microsoft ML, is there a particular format that the data is stored in, in source control?

[9:09] If the data is large and needs to be versioned, what are the current methods people are using?

[11:06] A word from Azure DevOps sponsor: Clear Measure.

[11:39] Some of the interesting architectures Damian has put together for machine learning.

[16:10] Damian walks Jeffrey through his machine learning model from source control to building, to packaging up the release, to deploying.

[19:20] For this type of model, where would be the physical environment where it's measuring information?

[20:24] Damian talks firewall rules, permissions, and security.

[23:16] The advantages of using Azure's IoT Hub.

[24:46] Damian talks about the new open source features that were added with the release.

[28:20] Does Damian still encounter customers who say they don't want to use Microsoft products because they don't realize they're open source?

[29:36] Is it true that VS Code is the most popular editor?

[31:03] One of the huge advantages of using open source.

[31:53] Damian talks build agents.

[33:33] About the new Windows-hosted container build agent.

[35:50] Damian's recommendation for listeners who want to go further with data science after listening to this week's podcast!

Mentioned in this Episode:

Azure DevOps

Azure Pipelines

Octopus Deploy

Clear Measure (Sponsor)

Buck Hodges on the introduction to Azure DevOps Services - Episode 001

Donovan Brown on How to Use Azure DevOps Services - Episode 002
Source control in Azure DevOps

Ubuntu

Machine Learning (ML)

Amazon Web Services (AWS)

Azure Cognitive Services

CustomVision.ai

Raspberry Pi
Azure Data Center

.NET Core

Python

GitHub

Azure IoT Hub

ADP Summit
VS Code

Docker Compose

Subversion

Chocolatey

Want to Learn More?

Visit AzureDevOps.Show for show notes and additional episodes.

Follow Up with Our Guest:

Damian Brady's LinkedIn


Download this episode (MP3, 37:33, 36.0 MB)

Watch this episode on YouTube

This episode on the AI DevOps Podcast site

Jeffrey Palermo

Lori Lamkin, Microsoft’s Director of PM on Shifting to Azure DevOps – Episode 007


Download this episode (MP3, 37:49, 36.3 MB)
Watch this episode on YouTube

In this episode, Jeffrey speaks with Lori Lamkin, Microsoft’s Director of Program Management. She has been leading the Visual Studio Team Services program management since the conception of Team Foundation Server in 2002 — that’s 16 years in the space of better shipping software using Microsoft technologies! She led the transition of the team to Agile methodologies, to open source reuse, to Cloud services, and Azure.

Lori and Jeffrey discuss what’s next for Lori in and of her role as Director of PM, her strategy behind leading the big shift from VSTS to Azure DevOps, the current roles and duties within Microsoft Azure DevOps, what she sees as the biggest shift in progressing from Agile and adopting DevOps, and how DevOps has become more and more efficient.

Topics of Discussion:

[:51] About today’s guest, Lori Lamkin!

[1:18] Jeffrey welcomes Lori to the podcast.

[3:22] Lori speaks about the strategy behind leading the big shift from VSTS to Azure DevOps.

[5:42] What’s next for Lori in and out of her role as Director of PM? What has she been up to?

[8:18] Lori gives some background on Azure DevOps history.

[15:37] Lori talks about the current roles and duties within Microsoft Azure DevOps.

[18:08] A word from Azure DevOps sponsor: Clear Measure.

[18:34] How the organization for Azure DevOps is structured much differently than many other organizations.

[20:38] What Lori thinks is the biggest shift in progressing from Agile and adopting DevOps.

[23:34] Why everyone in Lori’s team is a Software Development Engineer.

[25:08] Did Lori’s team used to have Systems Engineers (otherwise known as IT Pros)?

[27:18] The broad set of skills that is required of the developers to operate the components of Azure DevOps that the Azure Data Center is running on.

[28:55] The change in business with DevOps and how it has become more efficient.

[31:21] Lori’s take on the culture change CEO Satya Nadella is driving as well as his 1ES (one engineering system) plan.

[35:26] Is there no software Git can’t handle?

[36:18] Where Lori suggests you further your learning after listening to this episode.

Mentioned in this Episode:

Azure DevOps
Azure DevOps Services
Azure Cloud

Clear Measure (Sponsor)

Microsoft Secrets: How the World’s Most Powerful Software Company Creates Technology, Shapes Markets, and Manages People, by Michael A. Cusumano

Satya Nadella on transforming Microsoft’s culture

More about Satya’s One Engineering System initiative
TFVS

AKA.MS/DevOps

GitHub

Want to Learn More?

Visit AzureDevOps.Show for show notes and additional episodes.

Follow Up with Our Guest:

Lori Lamkin’s LinkedIn

Check out this episode!

Jeffrey Palermo

Palermo Pamphlet 002 – State machine design


Download this episode from the Internet Archive (MP4, 7:31, 113.1 MB). Its first host no longer has it.

In this episode, I go over how to design a workflow feature for an entity in a software application. It’s important to separate states from state transitions.

Here are the show notes

Jeffrey Palermo

Edward Thomson on All Things Git, libgit2, and Azure DevOps – Episode 006


Download this episode (MP3, 50:19, 48.3 MB)
Watch this episode on YouTube

This week, your host, Jeffrey Palermo, brings you a recording, live from the Microsoft Ignite conference. He interviews Edward Thomson, the Principal Program Manager for Azure DevOps, the maintainer of libgit2 (the git library), a host of All Things Git (a podcast about Git), and the curator of Developer Tools Weekly (a weekly newsletter with developer tool news). After the tight-knit company Edward was a Software Engineer for — Teamprise — was acquired by Microsoft, Edward transitioned into the role of Software Engineer with Microsoft, then GitHub briefly, 6 years later. After Github, Edward returned to Microsoft in 2017, this time as their Senior Program Manager — and most recently, as their Principal Program Manager (since two months ago).


Edward is a huge mover and shaker in the Git area of Azure DevOps Services. And in this episode, he answers questions that tons of teams continue to ask in the space. He also speaks about his many side projects: libgit2, All Things Git, and Developer Tools Weekly.

Topics of Discussion:

[1:11] About today’s episode and guest.

[1:37] Jeffrey welcomes Edward to the podcast.

[2:13] A bit about the Microsoft Ignite conference.

[3:31] How Edward landed the role he’s currently in, with Microsoft.

[6:38] Does the work Edward has been doing with Git, precede him joining Microsoft?

[9:58] A word from Azure DevOps sponsor: Clear Measure.

[10:24] What is libgit2? What does it do?

[13:11] Is there any source code that won’t work with Git?

[16:18] Version control of binary files — where does it all break down?

[18:11] Is there a file size that’s too large for Git?

[20:14] Does Edward recommend checking in your NuGet packages?

[24:12] When did the use of shallow cloning (for repositories) come about?

[26:44] What is Edward and the Azure DevOps team thinking about pushing forward with the Git engine (when Microsoft acquires GitHub)?

[30:03] Edward talks about his weekly newsletter, Developer Tools Weekly.

[31:29] What was in Edward’s weekly newsletter that he sent off just this morning (of recording this podcast).

[34:36] All about Edward’s podcast, All Things Git.

[36:27] Edward answers some FAQs in the Git area of Azure DevOps!

[47:03] Edward’s parting words for listeners on today’s podcast.

Mentioned in this Episode:

Microsoft Ignite

Microsoft Ignite’s Twitter
ADP: Buck Hodges episode

ADP: Donovan Brown episode
ADP: Sam Guckenheimer episode

libgit2

All Things Git Podcast

Developer Tools Weekly

Teamprise

Visual Studio

GitHub

Clear Measure (Sponsor)

VFSforGit

GitKraken

Sourcetree

Azure Repos

Game Developers Conference (GDC)

NuGet

Shallow Cloning
Azure Pipelines

dev.azure.com/libgit2

Want to Learn More?

Visit AzureDevOps.Show for show notes and additional episodes.

Follow Up with Our Guest:

Edward Thomson’s LinkedIn

Edward Thomson’s Twitter

Check out this episode!

Jeffrey Palermo

Palermo Pamphlet launch – episode 001


Download this episode (MP4, 4:41, 76.3 MB)

My goal is to teach, inform, and have a little fun. But I want this to provide value for programmers shipping custom software using Microsoft tools. Here is the first episode of the Palermo Pamphlet, which I hope will be a valuable resource to you.

Here are the show notes

Jeffrey Palermo

Performance tuning an Azure DevOps build configuration

We’ve all seen the culprits that constantly add time to builds.  One might observe that your NPM install or Nuget restore can take several minutes.  I remember back to the times of CC.Net in 2005 when a small application build could happen in 45 seconds, including unit tests.  And 10 minutes as a “thou shall not go over this” threshold.  So we cannot allow NPM or any other step to take minutes.  We have to ferret that out.
The answer is the same as code performance profiling.  Find out where every build is spending the same time doing work that adds no value or doesn’t vary often. Then we cache the result.  For so many builds, these are the culprits that take time but typically aren’t the changes that are being tested from build to build:
  1. Obtaining a build server (when choosing hosted build agents)
  2. Cloning the source
  3. Package restores
  4. Copying/archiving build artifacts
Here are my common solutions for reducing these common culprits (I’d be interested to know how others have eliminated these time sucks)
  1. Use our own Azure VMs as the build agents (running multiple agents on a single VM) – always available at a moments notice
  2. Let Azure Pipelines be a little less aggressive with cleaning source and instead have the build script delete the build directories at the beginning – removes need for a full clone and can just be a pull (works most of the time and requires probably a monthly purge for a clean clone, but saves SOOO much time)
  3. a) retain cloned working tree so that the previous package restore is used for subsequent builds or b) check in packages so that package restores are not necessary for every build
  4. Once builds are working and reliable, only archive the build artifacts that are directly used by the release pipeline (typically the nuget packages that house the application components)
Jeffrey Palermo

Dave McKinstry on Integrating Azure DevOps and the Culture of DevOps – Episode 005


Download this episode (MP3, 34:39, 33.3 MB)
Watch this episode on YouTube

This week, Jeffrey Palermo is joined by his guest, Dave McKinstry. Dave is a Program Manager with the Azure DevOps Services Community Team — connecting with partners and customers, spreading modern practises, and helping developers succeed with DevOps and Azure. Prior to his position at Microsoft, he has been in software services and technical sales for over 18 years. As a consultant, principal consultant, co-owner, and manager, he has always helped people efficiently build better software. He loves what he does as a technologist and enjoys being a part of today’s rapid technology evolution.

In this episode, Jeffrey and Dave talk about changes for Dave since the launch of Azure DevOps, what his journey has been like in the DevOps industry, his thoughts on companies looking to integrate Azure DevOps and move forward with automated deployment and reaching the continuous integration mark, how he thinks developers can move forward in terms of quality and Agile 101, and the modern skillset of what a developer and/or system engineer should look like in today’s DevOps environment.

Topics of Discussion:

[:39] About today’s guest, Dave McKinstry.

[1:00] Jeffrey welcomes Dave to the podcast.

[1:14] How it has been for Dave since the launch of Azure DevOps.

[1:25] Which side does Dave work on? Azure DevOps Services or Azure DevOps Server?

[2:58] Is Dave going to be at the Ignite event?

[3:10] What has been Dave’s journey through (what we now talk about as) modern practices and shipping software?

[5:25] How is it with Dave’s customers in regards to getting to continuous delivery and the continuous integration mark.

[8:03] The general nature of smaller and larger companies from a business perspective.

[10:03] Dave’s thoughts on the companies integrating Azure DevOps looking into moving forward with automated deployment.

[12:20] A word from Azure DevOps sponsor: Clear Measure.

[12:48] How developers can move forward with quality and Agile 101.

[15:10] How did the culture of DevOps (Dev and Ops) come together?

[17:57] Dave’s take on the relevant tasks of 15 years ago, no longer being relevant in today’s DevOps environment.

[20:28] The modern skillset of DevOps and what developers and system engineers need to be doing in the current DevOps world.

[21:29] About the benefits of Dave’s standing treadmill desk (that he’s currently using during the recording of the podcast)!

[25:40] Dave and Jeffrey’s early schooling, programming, and typing experiences! And the contrast with modern day schooling and the changing world.

[28:59] Dave describes the term ‘machine learning’ and the impact it has.

[33:15] What Dave recommends listeners should do next.

Mentioned in this Episode:

Azure DevOps

Azure DevOps Services

Azure DevOps Server

Microsoft Ignite event
Application Lifecycle Management (ALM)
Team Foundation Server (TFS)

CICD

Better Business Bureau

Clear Measure (Sponsor)

Agile 101

Mythical Man-Month: Essays on Software Engineering, by Fred Brooks

IoT

aka.ms/devops

Want to Learn More?

Visit AzureDevOps.Show for show notes and additional episodes.

Follow Up with Our Guest:

Dave McKinstry’s LinkedIn

Check out this episode!

Jeffrey Palermo

Why I started the Azure DevOps Podcast

I wanted to share a little bit of my reasoning for starting the Azure DevOps Podcast.  The above video is about 4 minutes long.  Please take a look.  Feel free to play at double speed.  The gist of it is I like seeing developers having fun.  Busted releases are not fun.  All-weekend deployments are not fun.  New things breaking when you make a code change are not fun.  And software development can be SO MUCH FUN!  I’ve spent 21 years so far as a professional programmer – not counting the toy apps and websites outside that time. I still love it, and I would like to share that joy with anyone I can.

Here are some links I mention in the video: