Chapter 17
GraphQL Advanced Concepts
Server Side Caching
Step 1: Adding End to End watch script to package.json file
Adding an End-to-End (E2E) watch script to the package.json file allows for the continuous observation and testing of the application as changes are made, facilitating immediate feedback and efficient issue detection. This script is configured to run the E2E testing suite in a watch mode, which monitors for file changes and automatically reruns tests, proving essential for iterative development and enhancing productivity.
The integration of a watch mode for E2E tests into the development workflow can significantly improve code quality by enabling developers to identify and resolve integration issues as they code. It acts as a real-time safeguard against regressions and defects, which is a reflection of high standards in software development and maintenance.
"test:e2e:watch": "jest --watch --detectOpenHandles --config ./test/jest-e2e.json"
Step 2: Create a new E2E Testing File
You can have to create a new testing file in the test/song/song.e2e-spec.ts
Step 3: Setup a E2E Testing File
import { Test, TestingModule } from "@nestjs/testing";
import { INestApplication } from "@nestjs/common";
import * as request from "supertest";
import { AppModule } from "../song/../../src/app.module";
import { TypeOrmModule } from "@nestjs/typeorm";
import { Song } from "../song/../../src/song/song.entity";
import { SongModule } from "../song/../../src/song/song.module";
import { CreateSongDTO } from "../song/../../src/song/dto/create-song-dto";
describe("Song Resolver (e2e)", () => {
let app: INestApplication;
beforeEach(async () => {
const moduleFixture: TestingModule = await Test.createTestingModule({
imports: [AppModule],
}).compile();
app = moduleFixture.createNestApplication();
await app.init();
});
beforeAll(async () => {
const moduleRef = await Test.createTestingModule({
imports: [
TypeOrmModule.forRoot({
type: "postgres",
url: "postgres://postgres:root@localhost:5432/test-dev",
synchronize: true,
entities: [Song],
dropSchema: true,
}),
SongModule,
],
}).compile();
app = moduleRef.createNestApplication();
await app.init();
});
afterEach(async () => {
const songRepository = app.get("SongRepository");
await songRepository.clear();
});
afterAll(async () => {
await app.close();
});
const createSong = (createSongDTO: CreateSongDTO): Promise<Song> => {
const song = new Song();
song.title = createSongDTO.title;
const songRepo = app.get("SongRepository");
return songRepo.save(song);
};
it("/ (GET)", () => {
return request(app.getHttpServer())
.get("/")
.expect(200)
.expect("Hello World!");
});
});
To ensure test accuracy and environment isolation, the Songs table in the database must be cleared after each test is executed. This practice prevents state leakage between tests and maintains consistency.
The file to initiate End-to-End testing in watch mode is executed with the command npm run test:e2e:watch. This initiates a process that continually monitors for any changes in the codebase and reruns tests accordingly, allowing for immediate feedback and efficient debugging.
Step 4: Test the Songs Query
it("(Query) it should get all songs with songs query", async () => {
const newSong = await createSong({ title: "Animals" });
const queryData = {
query: `query {
songs {
id
title
}
}`,
};
const results = await request(app.getHttpServer())
.post("/graphql")
.send(queryData);
expect(results.statusCode).toBe(200);
expect(results.body).toEqual({ data: { songs: [newSong] } });
});
Testing queries in Nest.js, especially when involving a database, begins by ensuring that the relevant records exist to be queried against. The setup phase of a test typically includes invoking methods like createSong to insert necessary data into the Songs table, resulting in a controlled and predictable testing environment.
The formation of GraphQL queries is facilitated through the use of structured variables, such as queryData, which hold the GraphQL query string. These queries are then utilized to interact with the GraphQL API, which, by convention, operates under a single endpoint, typically /graphql.
To execute these queries within a test suite, the query data must be provided to the send method of the testing tool. This simulates a client request to the server, allowing for the testing of query handling and response accuracy.
In high-quality software engineering, a tip is to ensure the testing environment closely mimics production conditions without affecting real data, maintaining isolation between tests for reliability. Additionally, structuring test cases to reflect a variety of realistic scenarios can uncover edge cases and potential bugs before deployment.
Step 5: Test A Single Song Query
it("(Query) it should get a song by id", async () => {
const newSong = await createSong({ title: "Animals" });
const queryData = {
query: `query GetSong($id: ID!){
song(id: $id){
title
id
}
}`,
variables: {
id: newSong.id,
},
};
const results = await request(app.getHttpServer())
.post("/graphql")
.send(queryData)
.expect(200);
expect(results.body).toEqual({ data: { song: newSong } });
});
Defining a Query in GraphQL with specific operation names and required variables, such as GetSong($id: ID!), clarifies the expected input and makes the query reusable. Variables in GraphQL, like $id, are used to parameterize queries, allowing for dynamic input that can be varied with each query execution.
Supplying the $id variable within a song query allows for the specification of which particular song is being requested. This is a part of query composition in GraphQL, where the query variables are declared and then used within the query body to fetch data.
The variables section in a GraphQL operation is where the actual values for the declared variables are specified. This is where one sets the specific ID value for $id when executing the query, thereby allowing for the retrieval of data corresponding to that particular identifier.
Step 6: Test The Creation of a aong mutation
it("(Mutation) it should create a new song", async () => {
const queryData = {
query: `mutation CreateSong($createSongInput: CreateSongInput!){
createSong(createSongInput: $createSongInput){
title
id
}
}`,
variables: {
createSongInput: {
title: "Animals",
},
},
};
const results = await request(app.getHttpServer())
.post("/graphql")
.send(queryData)
.expect(200);
expect(results.body.data.createSong.title).toBe("Animals");
});
Step 7: Testing The Update Song Mutation
The testing of the 'Create Song Mutation' in Nest.js involves invoking the mutation responsible for creating a new song entry and verifying that the operation successfully adds the song to the application state or database. The test ensures that the mutation receives the correct input and produces the expected song record, consistent with defined data schemas and business logic.
it("(Mutation) it should update existing song", async () => {
const newSong = await createSong({ title: "Animals" });
const queryData = {
query: `mutation UpdateSong($id: ID!, $updateSongInput: UpdateSongInput!){
updateSong(id: $id, updateSongInput: $updateSongInput){
affected
}
}`,
variables: {
id: newSong.id,
updateSongInput: {
title: "Lover",
},
},
};
const results = await request(app.getHttpServer())
.post("/graphql")
.send(queryData)
.expect(200);
expect(results.body.data.updateSong.affected).toBe(1);
});
Step 8: Test The Delete Song Mutation
Testing the 'Update Song Mutation' entails executing the mutation that modifies an existing song entry and confirming that the changes are accurately persisted. This test asserts that the mutation not only accepts valid input but also integrates seamlessly with the underlying data handling layers, effectively maintaining data integrity throughout the operation.
it("(Mutation) it should delete existing song", async () => {
const newSong = await createSong({ title: "Animals" });
const queryData = {
query: `mutation DeleteSong($id: ID!){
deleteSong(id: $id){
affected
}
}`,
variables: {
id: newSong.id,
},
};
const results = await request(app.getHttpServer())
.post("/graphql")
.send(queryData)
.expect(200);
expect(results.body.data.deleteSong.affected).toBe(1);
});
For optimal results, these tests would typically include checks for handling of invalid inputs and edge cases to ensure resilience. Leveraging a suite of such mutation tests establishes a safeguard against regressions and aids in maintaining a stable and reliable codebase as new features are developed.
Optimize Query Performance using DataLoader
What is DataLoader?
In GraphQL, a dataloader is a mechanism used to efficiently batch and cache requests made to a backend data source. It is particularly useful in scenarios where multiple GraphQL queries or mutations request the same or overlapping data, leading to redundant or redundant data-fetching operations. The dataloader helps to optimize these scenarios by reducing the number of database or API calls, resulting in improved performance and reduced latency.
The idea behind a data loader is to collect multiple individual data requests and then fetch the data in batches, minimizing the number of trips to the data source. It acts as a middleware or utility between the GraphQL resolver functions and the data source, sitting at the data access layer.
Here's how a data loader typically works:
Request Batching: When a resolver needs to fetch data from the data source, it registers the request with the dataloader instead of making the direct call immediately. The dataloader collects these requests during the GraphQL query execution phase.
Batch Execution: Once the execution phase is complete, the dataloader identifies identical or overlapping requests and groups them into batches. It then executes these batches in one or more optimized calls to the underlying data source.
Caching: The dataloader often incorporates a caching mechanism to store and reuse fetched data for future requests, further reducing the need for redundant data fetches.
By using a dataloader, GraphQL servers can avoid the "N+1" problem, where resolving N objects in a GraphQL query requires N+1 database or API calls. Instead, the dataloader can efficiently resolve multiple objects with a smaller number of data source calls.
Dataloader implementations are available for various programming languages and frameworks that support GraphQL, making it easier for developers to optimize data fetching in their GraphQL APIs. For example, in JavaScript, the dataloader library is commonly used for this purpose.
Run the Application without DataLoader
Running an application without DataLoader in a Nest.js GraphQL context can result in increased numbers of data retrieval operations, potentially leading to what is known as the "N+1" problem. Run the application to see what will happen.
type User {
id: Int!
name: String!
}
type Post {
id: String!
title: String!
body: String!
createdBy: User!
}
type Query {
posts: [Post!]!
users: [User!]!
}
{
posts{
id
title
body
createdBy{
id
name
}
}
}
Upon sending the specified request to the GraphQL server, the system initiates a database query for the createdBy field within the users table. DataLoader addresses the inefficiency by batching user IDs and caching them.
Execution of the aforementioned request results in console outputs such as FETCHING DATA FROM DB: Getting user with id 1... multiple times for the same user ID. This indicates repetitive fetching of user records for each post by the createdBy resolver. DataLoader mitigates this issue by batching requests and utilizing caching mechanisms.
Step 1: Create UsersLoader in the users folder
Ensure that the dataloader package is installed.
The creation of users.loader should take place within the user's directory.
import * as DataLoader from "dataloader";
import { mapFromArray } from "../util";
import { User } from "./user.entity";
import { UsersService } from "./users.service";
export function createUsersLoader(usersService: UsersService) {
return new DataLoader<number, User>(async (ids) => {
const users = await usersService.getUsersByIds(ids);
const usersMap = mapFromArray(users, (user) => user.id);
console.log("usersMap", usersMap);
const results = ids.map((id) => usersMap[id]);
console.log("results", results);
return results;
});
}
This specific setup creates a DataLoader that batches user retrieval by their identifiers, leveraging a service method that fetches users in bulk and a utility function to map the resulting array to an object for constant-time lookups. This pattern minimizes unnecessary database queries and is a resource-efficient strategy for fetching associated data, such as user information, by aggregating multiple queries into a single batch. This approach is reflected in the asynchronous function passed to the DataLoader constructor, which organizes the users into a map and resolves the batch of user requests in an optimized sequence that aligns with the initial ordering of IDs.
Step 2: Register Loader in Context
@Module({
imports: [
PostsModule,
GraphQLModule.forRootAsync({
imports: [UsersModule],
useFactory: (usersService: UsersService) => ({
autoSchemaFile: join(process.cwd(), 'src/schema.gql'),
context: () => ({
randomValue: Math.random(),
usersLoader: createUsersLoader(usersService),
}),
}),
inject: [UsersService],
}),
],
})
The above configuration within a @Module decorator is setting up a GraphQL module with asynchronous importation. This approach ensures that dependencies such as UsersModule are loaded correctly and their services, like UsersService, are available for use within the GraphQL context.
For efficient data fetching and batching, the usersLoader is instantiated through the createUsersLoader function, providing a DataLoader instance specifically for user data. This setup is vital for optimizing GraphQL query performance by reducing the number of data access requests, particularly beneficial in scenarios with complex data loading requirements such as nested queries.
Step 3: Load data from a loader in PostResolver
Loading data from a loader in the context of a Nest.js resolver, such as PostResolver, involves a key strategy for optimizing data retrieval when dealing with GraphQL queries. It leverages tools like DataLoader to batch and cache data requests, which significantly enhances performance by reducing the number of database queries. It's a recommended practice to implement this pattern when dealing with GraphQL resolvers to ensure efficient and scalable data fetching operations.
@ResolveField('createdBy', () => User)
getCreatedBy(
@Parent() post: Post,
@Context('usersLoader') usersLoader: DataLoader<number, User>,
) {
const { userId } = post;
return usersLoader.load(userId);
}
Step 4: Run the Application
In the console, you will observe the following output: Getting users with IDs (1, 2, 4). This output signifies that the DataLoader has efficiently consolidated multiple database requests into a single query, fetching all the users associated with each post, optimizing data retrieval and reducing database load.
Fetching Data from External API
What is RESTDatasource in Apollo
In Apollo Server, a RESTDataSource is a built-in class provided by the apollo-datasource-rest package that facilitates fetching data from RESTful APIs and integrating it with your GraphQL server. It acts as a data source for your GraphQL resolvers, allowing you to easily make HTTP requests to external REST APIs and transform the responses into GraphQL-compatible data.
The RESTDataSource class provides several benefits:
HTTP Requests: It abstracts away the details of making HTTP requests, including handling GET, POST, PUT, DELETE, etc., so you can focus on fetching data from the API.
Caching: It has built-in caching support, which can improve the performance of your GraphQL server by reducing redundant requests to the same REST API endpoints.
Error Handling: It automatically handles errors and HTTP status codes, making it easier to manage error responses from the REST API.
GraphQL Integration: It maps the fetched data to your GraphQL schema, allowing you to define resolvers that directly use the methods provided by RESTDataSource to fetch data.
We are going to fetch data from the jsonplanceholder api. We will this endpoint https://jsonplaceholder.typicode.com/todos to fetch all the todos.
Step 1: Create Schema file
type Todo {
id: ID!
userId: Int!
title: String!
completed: Boolean
}
type Query {
todos: [Todo!]!
}
Step 2: Implement RESTDatasource in TodoService
import { RESTDataSource } from "@apollo/datasource-rest";
import { Injectable } from "@nestjs/common";
@Injectable()
export class TodoService extends RESTDataSource {
constructor() {
super();
this.baseURL = "https://jsonplaceholder.typicode.com";
}
async getTodos() {
return this.get("/todos");
}
}
Step 3: Use DataSource in TodosResolver
@Resolver()
export class TodoResolver {
@Query("todos")
async getTodods(
@Context("dataSources")
dataSources
) {
return dataSources().todoAPI.getTodos();
}
}
Step 4: Add Datasource in context
GraphQLModule.forRoot<ApolloDriverConfig>({
driver: ApolloDriver,
typePaths: ['./**/*.graphql'],
definitions: {
path: join(process.cwd(), 'src/graphql.ts'),
outputAs: 'class',
},
context: async () => ({
dataSources,
}),
}),
$7 bundle of my best NestJS + backend developer Courses.
More details coming soon.
Get the latest insights from the marketing world.
