Using LocalStack for Development Environments

by Bogdan Bujdea

A year ago, I was faced with an interesting challenge after I just finished migrating a .NET Core app from Azure to AWS. The project was open-source and we already had contributors waiting for the migration to be ready so they can start working on issues. As an open-source maintainer, this is exactly what you’d dream of. However, once people started working on it, we realized the onboarding process was a nightmare.

We had documentation, and the steps were pretty simple… that is, if you were an experienced AWS developer. Basically, in order to run the .NET Core API, they would require an AWS account configured with certain permissions, but here is where problems started to appear.

Frontend developers just wanted the server to be up and running so they can test the integration between server and client (React.js). Once they hear about creating cloud accounts and configuring them… they lose their interest. And I really don’t blame them because they are volunteering for this, they are not getting paid. They are donating a few hours from their time to help us with our project, not to setup accounts and configure IAM permissions.

Then we also had backend developers who encountered all sorts of issues with the configuration. Either they forgot to create some group or role, or I forgot to add it to the documentation.

Soon I was getting tired of doing support for the onboarding process, so I started to look for a better way. From all the solutions I found, the most promising one was LocalStack, a fully functional local AWS cloud stack.

In a nutshell, LocalStack promised to fix all our onboarding issues, but was it up to the challenge? Well, I’ll leave the conclusion for the end, and for now, I’ll show you how to install LocalStack on your machine and configure a .NET project with it.

  1. What is LocalStack
  2. Installing LocalStack on your machine
  3. .NET Core setup
  4. Data visualization

What is LocalStack 

Basically, it’s the entire AWS cloud on your machine. You have every service from AWS at your disposal and you don’t even need an AWS account. But this is not the only advantage; there’s more:

  • Actual HTTP REST services: All services in LocalStack allow actual HTTP connections on a TCP port.
  • Language agnostic: It works well with arbitrary programming languages and environments, due to the fact that we are using the actual REST APIs via HTTP.
  • Error injection: LocalStack allows injection of the errors that frequently occur in real cloud environments.
  • Isolated processes: LocalStack services live in isolation as separate processes available via HTTP, which fosters true decoupling and more closely resembles the real cloud environment.
  • Pluggable services: All services in LocalStack are easily pluggable and replaceable due to the fact that they are using isolated processes for each service. So this means you can have parts of your app communicating with AWS, and some of them can use mocks provided by LocalStack.

Installing LocalStack on your machine

You can either install LocalStack using pip or using Docker. The installation docs provided by them are enough to help you regardless of the requirements or the OS you’re using.

In my project, I decided to use Docker, so I modified a bit the docker-compose file that they provided and I added the services required by my project(DynamoDB, S3, and SSM):

version: '3.7'

services:
  localstack:
    network_mode: bridge
    image: localstack/localstack:latest
    container_name: localstack
    environment:
     - SERVICES=s3:4572,ssm:4583,dynamodb:4569
     - DATA_DIR=/tmp/localstack/data
    ports:
     - "4563-4599:4563-4599"
     - "9999:8080"
    volumes:
      - localstack-data:/tmp/localstack

volumes:
  localstack-data:
    name: localstack-data

Once you save this as “docker-compose.yaml”, just open a terminal in the same folder and run: docker-compose -d

Congratulations! In a few seconds, the Docker image should be downloaded and LocalStack will be up and running on your machine.

Integration with .NET Core

I’ll give you 3 examples of how I used this in my project, with the services that I mentioned above: DynamoDB, S3, and SSM.

DynamoDb

In .NET Core projects, the easiest way to use DynamoDb is to install the AWSSDK.DynamoDBv2 Nuget package, and add this line to your Startup.cs file: services.AddAWSService<IAmazonDynamoDB>();

Based on the configuration provided, the DynamoDb client would know which region and credentials to use. In order to use the LocalStack service for DynamoDb, we’re going to use the same library, but we’re going to manually specify the configuration so it will use the DynamoDb endpoint from LocalStack:

if (_environment.IsDevelopment())
{
    services.AddSingleton<IAmazonDynamoDB>(sp =>
    {
        var clientConfig = new AmazonDynamoDBConfig
        {
            ServiceURL = "http://localhost:4569",
            UseHttp = true
        };
        return new AmazonDynamoDBClient(new BasicAWSCredentials("abc", "def"), clientConfig);
    });
    Console.WriteLine("Added LocalStack DynamoDb");
}
else
{
    services.AddAWSService<IAmazonDynamoDB>();
}

As you can see, we’re using the same Nuget package to connect to LocalStack on Development. For other environments, we use the real service.

If you’re having issues, make sure you also set the “UseHttp” property to true, and if you’re running on Windows, then instead of “localhost” you should be using “host.docker.internal”.

http://localhost:4569

becomes

http://host.docker.internal:4569

S3

If we also have S3 in our project, the process is basically the same. We need the AWSSDK.S3 Nuget package, and depending on the environment, we’ll decide which service to use, the real one or LocalStack.

if (_environment.IsDevelopment())
{
    var amazonS3 = new AmazonS3Client(new BasicAWSCredentials("abc", "def"), new AmazonS3Config
    {
        ServiceURL = "http://localhost:4572",
        ForcePathStyle = true,
        UseHttp = true
    });

    services.AddSingleton(typeof(IAmazonS3), provider => amazonS3);
    Console.WriteLine("Added LocalStack S3");
}
else
{
    services.AddAWSService<IAmazonS3>(new AWSOptions
    {
        Profile = "default"
    });
}

SSM

For SSM, the changes will be in the Program.cs file. The logic is the same; we’ll decide which service to use depending on the environment.

public static IWebHostBuilder CreateWebHostBuilder(string[] args)
{
    return WebHost.CreateDefaultBuilder(args)
        .ConfigureAppConfiguration((hostingContext, builder) =>
        {
            var env = hostingContext.HostingEnvironment;
            var configurationBuilder = builder.AddJsonFile("appsettings.json", true, true)
                .AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: true,
                    reloadOnChange: true);
            if (env.IsDevelopment())
            {
                builder.AddSystemsManager($"/{Consts.ParameterStoreName}-dev", new AWSOptions
                {
                    DefaultClientConfig =
                            {
                                ServiceURL = "http://localhost:4583",
                                UseHttp = true
                            },
                    Credentials = new BasicAWSCredentials("abc", "def")
                }, TimeSpan.FromSeconds(5));
                Console.WriteLine("Added LocalStack SSM");
            }
            else
            {
                builder.AddSystemsManager($"/{Consts.ParameterStoreName}", new AWSOptions(),
                    TimeSpan.FromSeconds(30));
            }
        }).UseStartup<Startup>();
}

In my project, I had to add some values in the parameter store that would be the same for every developer. For example, I had to store a number that would define the refresh rate in seconds for our app. For development purposes, this doesn’t really matter, but in production, we had to be able to change this without restarting the app, so SSM provided exactly what we were looking for.

Once LocalStack is started, it will be empty of course, so we should add our data in the parameter store before the API is started. For this, you can use the command line and invoke LocalStack just as you would invoke the AWS CLI. The only difference is that here you’re going to specify the LocalStack endpoint, so AWS CLI will call the LocalStack service instead of the real one.

Here’s an example:

aws --endpoint-url=http://localhost:4583 ssm put-parameter --name "/my-app-param-store/settings/intervalInSeconds" --type String --value "60" --overwrite --region "us-east-1"

This is a regular AWS CLI command; maybe the only difference is that usually you don’t specify the “-endpoint-url” parameter. The advantage here is that you don’t have to learn something new in order to use the CLI. What’s more, you can create a script for every OS that can be executed by developers before running the project.

I actually did two scripts, a “setup.cmd” and a “setup.sh” that a developer could execute and have LocalStack running and configured. Here’s how it looked like for Windows:

docker-compose up -d
aws --endpoint-url=http://localhost:4583 ssm put-parameter --name "/my-app-param-store/settings/intervalInSeconds" --type String --value "60" --overwrite --region "us-east-1" &

This script was placed in the same folder as the docker-compose file. The first line would start the Docker image for LocalStack, and the next line would add a parameter in the store. I actually modified the docker-compose file even further and placed the .NET Core project in there as well, so a frontend developer could run this script and have LocalStack + our API running in under a minute.

Data visualization

As I said in the beginning, LocalStuck is a fully functional AWS cloud, not just a mock. This means that you can really have S3 buckets and DynamoDb tables on your machine. But once you added some data in LocalStack, how exactly do you visualize it?

There are two ways. First, there’s the web interface provided by LocalStack. In our docker-compose file, you can see that we exposed the port 9999, so you can navigate to localhost:9999 and see the LocalStack web app.

AN overview of deployed resources on aws.

However, my favorite way of doing this is with an app called Commandeer. It has a nice UX and it allows you to control LocalStack from it.

Conclusion 

Was it up to the challenge? Definitely! It made the onboarding process a lot easier, from spending hours to configure your AWS account to running a Docker image in less than a minute.

Were there issues? Definitely, but they were easy to fix. Most problems came from people that couldn’t install Docker on their system (Windows Home edition) or they had a faulty Python installation. Another issue was the LocalStack setup that I did in the beginning, where I spent a few hours until I realized that on Windows you have to use “host.docker.internal” instead of “localhost”, or that you must set the “UseHttp” property. Luckily for you, these tips are already in this article so you should be ready to go.

Considering the pros and cons, if you’re a team that wants to make the onboarding process a bit easier or you want to test your integration with AWS services but without actually paying for them, then LocalStack should be the right tool for you.

I hope this gave you an overview of LocalStack enough the get you started with it. To make this easier, I also made a LocalStack demo which is hosted on GitHub. Enjoy!

Bogdan Bujdea, .NET developer at Maxcode

About Bogdan Bujdea

Bogdan is a .NET developer and Microsoft MVP on Developer Technologies. He first got in touch with .NET 10 years ago, and it was love at first sight. In his free time, he likes to contribute to open source projects, get involved in the local .NET community, or implement some new features for his smart home and share it on his blog.

Share this article