Getting Localstack Up and Running

In continuation to the earlier post on mocks for clouds, this article does a deep dive into getting up & running with Localstack. This is a consolidation of the steps & best practices shared here, here & here. The Localstack set-up is on a Ubuntu-20.04, with Java-8x, Maven-3.8x, Docker-24.0x. 

(I) Set-up

    # Install awscli
     sudo apt-get install awscli

    # Install localstack ver 3.8
        ## Issue1: By default pip pulls in version 4.0, which gives an error:
        ## ERROR: Could not find a version that satisfies the requirement localstack-ext==4.0.0 (from localstack) 

        python3 -m pip install localstack==3.8.1

--

    # Add to /etc/hosts
    127.0.0.1    localhost.localstack.cloud
    127.0.0.1    s3.localhost.localstack.cloud

--

    # Configure AWS from cli
    aws configure
    aws configure set default.region us-east-1
    aws configure set aws_access_key_id test
    aws configure set aws_secret_access_key test

    ## Manually configure AWS
    Add to ~/.aws/config
    endpoint_url = http://localhost:4566

    ## Add mock credentials
    Add to ~/.aws/credentials
    aws_access_key_id = test
    aws_secret_access_key = test

--

    # Download docker images needed by the Lambda function
        ## Issue 2: Do this before hand, Localstack gets stuck
        ## at the download image stage unless it's already available

    ## Pull java:8.al2
    docker pull public.ecr.aws/lambda/java:8.al2

    ## Pull nodejs (required for other nodejs Lambda functions)
    docker pull public.ecr.aws/lambda/nodejs:18

    ## Check images downloaded
    docker image ls

(II) Start Localstack

    # Start locally
    localstack start

    # Start as docker (add '-d' for daemon)
       ## Issue 3: Local directory's mount should be as per sample docker-compose

    docker-compose -f docker-compose-localstack.yaml up

    # Localstack up on URL's
    http://localhost:4566
    http://localhost.localstack.cloud:4566

    # Check Localstack Health
    curl http://localhost:4566/_localstack/info
    curl http://localhost:4566/_localstack/health

(III) AWS services on Localstack from CLI

  (a) S3
    # Create bucket named "test-buck"
    aws --endpoint-url=http://localhost:4566 s3 mb s3://test-buck

    # Copy item to bucket
    aws --endpoint-url=http://localhost:4566 s3 cp a1.txt s3://test-buck

    # List bucket
    aws --endpoint-url=http://localhost:4566 s3 ls s3://test-buck

--

  (b) Sqs
    # Create queue named "test-q"
    aws --endpoint-url=http://localhost:4566 sqs create-queue --queue-name test-q

    # List queues
    aws --endpoint-url=http://localhost:4566 sqs list-queues

    # Get queue attribute
    aws --endpoint-url=http://localhost:4566 sqs get-queue-attributes --queue-url http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/test-q --attribute-names All

--

  (c) Lambda
    aws --endpoint-url=http://localhost:4566 lambda list-functions

    # Create Java function
    aws --endpoint-url=http://localhost:4566 lambda create-function --function-name test-j-div --zip-file fileb://original-java-basic-1.0-SNAPSHOT.jar --handler example.HandlerDivide::handleRequest --runtime java8.al2 --role arn:aws:iam::000000000000:role/lambda-test

    # List functions
    aws --endpoint-url=http://localhost:4566 lambda list-functions

    # Invoke Java function
    aws --endpoint-url=http://localhost:4566 lambda invoke --function-name test-j-div --payload '[200,9]' outputJ.txt

    # Delete function
    aws --endpoint-url=http://localhost:4566 lambda delete-function --function-name test-j-div

(IV) AWS services on Localstack from Java-SDK

    # For S3 & Sqs - localstack-aws-sdk-examples, java sdk

    # For Lambda - localstack-lambda-java-sdk-v1

 

To Mock a Cloud

Cloud hosting has been the norm for a while now. Saas, Paas, Iaas, serverless, AI whatever the form may be, organizations (org) need to have a digital presence on the cloud. 

Cloud vendors offer hundreds of features and services such as 24x7 availability, fail-safe, load-balanced, auto-scaling, disaster resilient distributed, edge-compute, AI/ Ml clusters, LLMs, Search, Database, Datawarehouses among many others right off-the-shelf. They additionally provide a pay-as-you-go model only for the services being used. Essentially everything that any org could ask for today & in the future!

But it's not all rosy. The cloud bill (even though pay-as-you-go) does burn a hole in the pockets. While expenses for the live production (prod) environment is necessary, costs for the other dev, test, etc, internal environments could be largely reduced by replacing the real Cloud with a Mock Cloud. This would additionally, speed up dev and deployment times and make bug fixes and devops much quicker & streamlined.

As dev's know mocks, emulators, etc are only as good as their implementation - how true they are to the real thing. It's a pain to find new/ unknown bugs on the prod only because it's an env very different from dev/ test. Which dev worth his weight in salt (or gold!) hasn't seen this ever?

While using containers to mock up cloud services was the traditional way of doing it, a couple of recent initiatives like Localstack, Moto, etc seem promising. Though AWS focussed for now, support for others are likely soon. Various AWS services like s3, sns, sqs, ses, lambda, etc are already supported at different levels of maturity. So go explore mocks for cloud & happy coding!

Mocking AWS ELB Behaviour Locally For Testing

Once hosted out of Amazon, you make use of the AWS Elastic Load Balancer (ELB) for balancing load across your EC2's within or acroos Availability Zones (AZ). Since code gets developed and tested locally (outside of Amazon), at times you might want to test load balancer scenarios before deploying to production. Here's one way to mock up the load balancer behaviour for local testing.

Use Apache (you could very well use something like Nginx instead) in a reverse proxy, load balancer set up via mod_proxy & mod_proxy_balancer.  Fairly simple for anyone with slight experience with configuring Apache. We used Apache as a load balancer front-end to IIS on local, exactly the way ELB would load balance in front of production IIS.

Additionally, since ELB was also an SSL end point for our production servers, we set up Apache to be the SSL end point (via mod_ssl) on local. Apache was configured to listen on port 443 (using a self-signed certificate), and would forward all traffic from port 443 to backend IIS on port 80.

Once we had that set-up going, we were quickly able to reproduce an issue with application generated Secure cookies not getting set properly across client request/ response. Once we had the fix on the local (which was to set the flag on the cookies in the request, not response) the same worked flawlessly on the AWS as well.