Wormwood - An Explicit Way to Test Absinthe GraphQL APIs

    We love GraphQL at Tinfoil! We use it extensively in our Elixir and Phoenix powered API scanner. We try to test out code using ExUnit whenever possible to help ensure a stable and smooth development cycle. Testing an Absinthe GraphQL API usually follows the pattern of: Setting up a ConnCase, making the request, and then validating that the data returned from the request was valid. A lot like the following:

    This works well! But as a result we may accumulate a whole lot of boilerplate code for setting up these Plug Conns and parsing their results. Not to mention the frequent module attributes and strings used to contain our GraphQL queries inside the unit test module itself. We figured there’s probably a better way to write these sorts of tests that can leverage the power of the Absinthe library itself, rather than sending HTTP requests during a test run.

    After some experimentation we created Wormwood, a small and open source Elixir library to assist with Absinthe GraphQL document testing in ExUnit. We can eliminate large chunks of Plug.Conn boilerplate, remove static strings of query code, explicitly scope a module to a single query document and schema, and call our GraphQL API like this instead:

    In the above snippet, Wormwood is loading, parsing and validating the query document at compile time, then, it runs that loaded query against the specified schema using Absinthe itself. With this method of GraphQL testing, it’s super explicit which query document we are testing, it’s also clear which schema we are executing it against. We also gain the benefit to test against the errors that Absinthe can return at the different phases of the pipeline, and even control the pipeline itself! (More on that later.)

Using Wormwood in Your Own App

    We’ll break down each of the steps for utilizing Wormwood in your own testing setup. You can get (and contribute to!) Wormwood on our GitHub! Wormwood is also available on the Hex package repository. Some of the example code shown in this post is available in the repo itself.

    The first requirement for using Wormwood is to break out your queries into individual GQL files. While this may sound excessive, it offers a lot of power in terms of code coverage, and project organization. You can read more about using individual GraphQL files with Webpack in the Apollo docs, it’s pretty simple to set up. Once you have all your documents broken out into files, we can select the ones we want to test, and make accompanying ExUnit test modules for them.

Let’s say we have this GQL document:

And our imported fragment is just a simple set of reusable attributes, it looks like this:

We can then create a basic ExUnit test module we’ll call “get_users_test.exs”, and we’ll load our schema and the GQL document into the module using the “load_gql/2” macro.

Let’s break down what Wormwood is doing here:

    When “load_gql/2” is executed it attaches two special  attributes to the module it was called from, they are assigned the Absinthe schema from the first argument, and the full source text of the query file from the second argument. Wormwood will expand all import statements it can find, and will raise an exception if it cannot find a file or if it could not validate the syntax of the full query with Absinthe.

Now that our module has a schema and document assigned, we can query it using the simple “query_gql/1” function:

Our query results now live in the “query_data” variable. If we inspect it, we can see that our results are returned in vanilla Elixir lists and maps:

    If you have a deeply nested structure, a good tip is to use the Elixir Kernel function “get_in\2”, which takes the structure you want to access data from, and a list of keys or access functions to retrieve specific members. For example, if I wanted to fetch the id of the first user in this big query result I could simply do the following:

More Advanced Queries

    The above example is just a simple demo of how to use Wormwood in your testing suite. Wormwood supports a few more options and configurations when testing. Below is a quick list of the features, along with snippets to show how it’s done. You can also dig around in the examples folder on the GitHub repo.

Running a Query With Variables and Context

   You can pass the same options keyword list you would pass to “Absinthe.run/3” into the “query_gql/1” function. Refer to the Absinthe docs on the exact options, and their usage. If we wanted to pass a variable into this query we can just leverage the options Absinthe provides:

The same can be done for context if you want to fake something like authentication:

Running a raw string, rather than a GQL file

    Of course if you don’t want to break out your GQL documents into files, you can still assign them to a test module as a raw string. Rather than calling “load_gql/2”, call “set_gql/2”:

Wormwood will still expand import statements when using raw strings! They will be relative to the current working directory, which is usually your app root directory.

Running a Query With a Custom Absinthe Pipeline

    If at any point you wish to modify the pipeline that Absinthe uses for executing a document loaded into a module, you can do so by composing a list of pipeline Phases, and passing them into the “query_gql_with_pipeline/2” function like so:


   Wormwood was born out of specific quirks we ran into while testing our API Scanner, we hope you find it useful as well! It aims to help accelerate and improve the way GraphQL tests are written within ExUnit. If you have stars, issues or contributions, feel free to leave any of them on the official GitHub repo!

Aaron Shea

Aaron is our Resident Robot, and fittingly he's crazy about automation. He's always investigating new ways to improve a developer's workflow. When he's not forging code tools, he's probably researching graphics and animation. Aaron studied Computer Science at University of Hartford.

Tags: graphql

Announcing GraphQL Security Scanning

For the second time this year: API security scanning changes today. We’ve been working hard on adding support to scan GraphQL APIs for security vulnerabilities, best practices, and correctness. Earlier this year, the Tinfoil Security API Scanner initially launched with support for the Swagger documentation format, and we’re excited to expand coverage to now include GraphQL APIs. To be clear, we’re not deprecating support for OpenAPI scanning - in fact, OpenAPI specification v3 support is coming soon! We’ve enjoyed building our own GraphQL APIs to power our user interfaces and we felt the need to ensure their correctness as we built them. To that end we’ve added first-class GraphQL support to our API Scanner. We’re thrilled today to announce the beta of our GraphQL scanning capabilities at the GraphQL Summit conference in San Francisco.

We use GraphQL internally to iterate quickly with our user interfaces without huge changes to the backend server each time. (We’re hiring, if Elixir, GraphQL, and Vue are interesting to you, by the way). GraphQL makes it easy to decouple user interface needs from a backend API server by offering a buffet of data and relationships without restricting the format to a specific JSON payload. Nowadays UI developers can iterate quickly, but this puts extra load on API server engineers to make a performant, and most importantly safe, GraphQL API.

One huge advantage of GraphQL APIs is that they are self-documenting. Most GraphQL APIs can be introspected to pull out the types, fields, and mutations. This can make it a joy to work with a tool like GraphiQL to explore an API, but also makes it very easy to get started scanning. All you need to do is provide the GraphQL endpoint and the Tinfoil Security API scanner will do the rest. We automatically discover the different types, fields, arguments, and mutations exposed by your API, and generate an optimized set of documents to exercise all of the different aspects of your API.

In addition to searching for various injections (both direct and blind), we also look for GraphQL-specific concerns. One such concern is cycles in the query graph, potentially DoSing an API server with a request that is time consuming to fulfill. GraphQL allows you to set complexity limits on documents received from the client to help prevent this, and our scanner makes sure your API server has a reasonable complexity limit set. When not auditing high-complexity queries, we make sure the documents we generate balance simplicity with API coverage.

Our support for GraphQL is only beginning; please stay tuned for more developments! If you’re interested in joining the beta of our GraphQL Scanning, please drop us a line.

Ben Sedat

Ben Sedat is the Engineering Wizard of Tinfoil Security. He's a bit of a blend between a traditional software engineer (builder) and security engineer (breaker). He spends a lot of time thinking about security: both detection as well as creating solutions for the security issues that exist in software and the internet. He also plays lots of video games. Lots.

Tags: graphql

Server-side GraphQL Querying with Elixir Absinthe

GraphQL is a few years old, and its promises are well known and pretty compelling. Get only the data your front end needs to display, introspection and type constraints, relate all your data in a graph of relationships, etc.! All great things, but if you’re like us and you start to retrofit a GraphQL API onto a REST-based site, you start to notice a divide.

We decided to build GraphQL into our API scanner. Since we’ve built it using Phoenix in Elixir, Absinthe was our go-to choice for a GraphQL query engine. Where before we had a set of contexts and related queries to provide information for our views, now we also had our GraphQL schema defining relationships and queries for fetching particular sets out of the database. It’s not a huge increase in maintenance and overhead, but it does mean duplicating authorization checks and a few other concerns, like remembering to preload for particular edge cases. It would be nice if we could use our GraphQL interface on the server side, particularly if you want to, say, pre-render a single-page app… and it turns out with Absinthe, you can!

Let’s take, for example, a simple social media site. On the site, there are users and posts, where users can become friends and posts can be liked. In order to populate the initial view of this site, we would need to get the current user, preload their friends, preload the first N posts between their posts and their friends’ posts, and the likes on those posts. We'd also need to have a GraphQL query for that same information when the state changes for the current user (for instance when they scroll to the bottom of the page). This is a good amount of duplicate querying, but with Absinthe you can add the @graphql annotation before a method in your controller to query the same information that your front end would pull. The results of the query becomes the parameters map given to your controller. For instance:

Relatively compact, fairly convenient, and by nesting everything under the current user, we should be able to ensure they only access things they are allowed to see. However, it looks like we’re grabbing just about every field available on our (admittedly quite simple) GraphQL schema. Also this query will return a bare map, rather than the structs defined in our application (which could be useful to have elsewhere in the app). Absinthe comes to the rescue for both of these issues by providing a shortcut in the @graphql annotation. Given a query where a field is requested but none of its subfields are specified, it will grab all the fields and, in the case of a field backed by an Ecto schema, will use that struct instead of a bare map. From there, you can use @put inside that object to grab the associations you want to load. This leaves us with this fairly succinct query for our controller’s index action:

And there we go! Now this hypothetical app need only worry about one path for providing data to users, and can concentrate on authorization along the GraphQL path. As long as our graph is complete, we don’t need to worry about making new specific queries for our controllers. Happy coding!

Alex Bullen

Alex is Tinfoil Security's Top-Shelf Programmer (and fetcher of things from high shelves). A former psychology wonk and recent App Academy grad, Alex endeavors to treat every challenge as an opportunity to improve his code-fu. When not busily building blocks of precisely put code, you can find him reading fantasy novels or practicing kung fu.

Tags: graphql