Using Amazon SQS directly from Amazon API Gateway

big_queue_3

A project I’m working on right now uses Amazon’s SQS service to handle interprocess communication between elements of a large and complex back end architecture.  SQS is a fully managed message queuing service that enables you to decouple and scale microservices, distributed systems, and serverless applications. Using SQS, you can send, store, and receive messages between software components at any volume, without losing messages or requiring other services to be available.

One of our uses cases requires a SQS message to be triggered and created by an external call to our API. The most common way of handling this is to use a Lambda function to respond to an API request and then the Lambda function will create the appropriate SQS message and add it to the queue. What if we could skip Lambda all together? Amazon API Gateway can now be used to directly created SQS messages.

High level steps I followed

  1. Created an Amazon SQS queue.
  2. Created an Amazon IAM policy and role (to access the SQS queue).
  3. Published an API on API Gateway.
  4. Send and receive messages (single and batch) through the API Gateway.

<>

Details on #3 (Published an API on API Gateway)

  • created a new api at https://us-west-2.console.aws.amazon.com/apigateway/
  • created resources /v1/send/
  • created GET method /v1/send/GET
  • added URL query string ‘MessageBody’ to method execution for GET
  • added Integration request that points to appropriate SQS ARN
    • Integration type: AWS Service
      AWS Region: us-west-2
      AWS Service: Simple Queue Service (SQS)
      AWS Subdomain: NULL
      HTTP method: GET
      Path override: 453862545997/ninegeartest (SQS ARN)
      Execution role: arn:aws:iam::453862545997:role/apigateway-sqs-ninthgear-access-role
      Credentials cache: Do not add caller credentials to cache key
      Content Handling: Passthrough
      Use Default Timeout

      URL Query String Parameters:

      Action ‘SendMessage’
      MessageBody method.request.querystring.MessageBody

      MessageBody=TestMessageHello

  •  

    Used API Gateway Test function to call /v1/send/GET with MessageBody and then confirmed it appeared correctly in the previously created queue

    Response is

    {
    “SendMessageResponse”: {
    “ResponseMetadata”: {
    “RequestId”: “dfc617e1-b3c9-56a8-b9bf-426457676e2d”
    },
    “SendMessageResult”: {
    “MD5OfMessageAttributes”: null,
    “MD5OfMessageBody”: “0ef0d7773a388095bee26966144cc131”,
    “MessageId”: “3eb7ed29-c259-4b15-8bd3-bedeac865ee5”,
    “SequenceNumber”: null
    }
    }
    }

    Logs

    Execution log for request 01e665e2-9394-11e9-80e3-7b3c608a2b3c
    Thu Jun 20 19:45:50 UTC 2019 : Starting execution for request: 01e665e2-9394-11e9-80e3-7b3c608a2b3c
    Thu Jun 20 19:45:50 UTC 2019 : HTTP Method: GET, Resource Path: /v1/send
    Thu Jun 20 19:45:50 UTC 2019 : Method request path: {}
    Thu Jun 20 19:45:50 UTC 2019 : Method request query string: {MessageBody=TestMessageHello}
    Thu Jun 20 19:45:50 UTC 2019 : Method request headers: {}
    Thu Jun 20 19:45:50 UTC 2019 : Method request body before transformations:
    Thu Jun 20 19:45:50 UTC 2019 : Endpoint request URI: https://sqs.us-west-2.amazonaws.com/453862545997/ninegeartest?Action=SendMessage&MessageBody=TestMessageHello
    Thu Jun 20 19:45:50 UTC 2019 : Endpoint request headers: {Authorization=*************************************************************************************************************************************************************************************************************************************************01fa36, X-Amz-Date=20190620T194550Z, x-amzn-apigateway-api-id=mf9zjl6bxj, Accept=application/json, User-Agent=AmazonAPIGateway_mf9zjl6bxj, X-Amz-Security-Token=AgoGb3JpZ2luEK///////////wEaCXVzLXdlc3QtMiKAAkhESmjWUuH4MmwXcX3mJZu1ZFF29by1qO8Y/mUjx1llmQ6VlpkceQ2wnYfz8Gr4loy/CxqjjQeKa4Zpz83c1c5aEV7LrX4XlBIFxH5c+0O/BHA4Y6JbcSCSzc2z/Y6AUpvBnrfdE+S7TAqQw0aXzDTVfoFiysOFhatFmGfDa3TmXbNje6yqsj2QJKcTpI2D8KlN1IqUcuIMyYW06j8iZlcDcDuaB9BNtw3cNimdZpvUknaOr6HZ9kaSmhhIGnyRYxTN7bEnsqEGQY00p1Xsr8Q8JTrAaiHYSZA+964bBPWbladcrBPPvYGw111yV8teyabwJ6BczFmPHONyKNBSEbwqigMIlf//////////ARAAGgw0NTM4NjI1NDU5OTciDJAFb/8TLGKSTv3VkireAptIn3MrBh0o+01/ckMLSEM8kpHeFlnXhWeqfbQ11PIEH36/I7ahYyZE0Z7eS8fGbZ4DQmXDjB5D2Z6Y8Wo8b1gfnxZ7MwGdYhDlo/x4KeE7R92XbhPX+tAEsL61TMx6Pls7c9 [TRUNCATED]
    Thu Jun 20 19:45:50 UTC 2019 : Endpoint request body after transformations:
    Thu Jun 20 19:45:50 UTC 2019 : Sending request to https://sqs.us-west-2.amazonaws.com/453862545997/ninegeartest?Action=SendMessage&MessageBody=TestMessageHello
    Thu Jun 20 19:45:50 UTC 2019 : Received response. Status: 200, Integration latency: 18 ms
    Thu Jun 20 19:45:50 UTC 2019 : Endpoint response headers: {x-amzn-RequestId=dfc617e1-b3c9-56a8-b9bf-426457676e2d, Date=Thu, 20 Jun 2019 19:45:50 GMT, Content-Type=application/json, Content-Length=276}
    Thu Jun 20 19:45:50 UTC 2019 : Endpoint response body before transformations: {“SendMessageResponse”:{“ResponseMetadata”:{“RequestId”:”dfc617e1-b3c9-56a8-b9bf-426457676e2d”},”SendMessageResult”:{“MD5OfMessageAttributes”:null,”MD5OfMessageBody”:”0ef0d7773a388095bee26966144cc131″,”MessageId”:”3eb7ed29-c259-4b15-8bd3-bedeac865ee5″,”SequenceNumber”:null}}}
    Thu Jun 20 19:45:50 UTC 2019 : Method response body after transformations: {“SendMessageResponse”:{“ResponseMetadata”:{“RequestId”:”dfc617e1-b3c9-56a8-b9bf-426457676e2d”},”SendMessageResult”:{“MD5OfMessageAttributes”:null,”MD5OfMessageBody”:”0ef0d7773a388095bee26966144cc131″,”MessageId”:”3eb7ed29-c259-4b15-8bd3-bedeac865ee5″,”SequenceNumber”:null}}}
    Thu Jun 20 19:45:50 UTC 2019 : Method response headers: {X-Amzn-Trace-Id=Root=1-5d0be26e-b7b766dba15eb99009933350, Content-Type=application/json}
    Thu Jun 20 19:45:50 UTC 2019 : Successfully completed execution
    Thu Jun 20 19:45:50 UTC 2019 : Method completed with status: 200

    <>

    More details and an example I followed at https://dzone.com/articles/creating-aws-service-proxy-for-amazon-sqs

     

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s