Shader Compilation

Shader Compilation is the process of text in the OpenGL Shading Language and loading it into OpenGL to be used as a Shader. OpenGL has three ways to compile shader text into useable OpenGL objects. All of these forms of compilation produce a Program Object.

Note: This article will contain references to OpenGL 4.x features, such as tessellation shaders or Compute Shaders. If you are using OpenGL 3.x, you can ignore such references.

Shader and program objects

A Program Object can contain the executable code for all of the Shader stages, such that all that is needed to render is to bind one program object. Building programs that contain multiple shader stages requires a two-stage compilation process.

This two-stage compilation process mirrors the standard compile/link setup for C and C++ source code. C/C++ text is first fed through a compiler, thus producing an object file. To get the executable code, one or more object files must be linked together.

With this method of program creation, shader text is first fed through a compiler, thus producing a shader object. To get the executable program object, one or more shader objects must be linked together.

Shader object compilation

The first step is to create shader objects for each shader that you intend to use and compile them. To create a shader object, you call this function:

GLuint glCreateShader(GLenum shaderType​);

This creates an empty shader object for the shader stage given by given shaderType​. The shader type must be one of GL_VERTEX_SHADER, GL_TESS_CONTROL_SHADER, GL_TESS_EVALUATION_SHADER, GL_GEOMETRY_SHADER, GL_FRAGMENT_SHADER, or GL_COMPUTE_SHADER. Note that the control and evaluation shaders require GL 4.0 (or ARB_tessellation_shader), and the compute shader requires GL 4.3 (or ARB_compute_shader).

Once you have a shader object, you will need to give it the actual text string representing the GLSL source code. That is done via this function:

void glShaderSource(GLuint shader​, GLsizei count​, const GLchar **string​, const GLint *length​);

This function takes the array of strings, given by string​ and stores it into shader​. Any previously stored strings are removed. count​ is the number of individual strings. OpenGL will copy these strings into internal memory.

When the shader is compiled, it will be compiled as if all of the given strings were concatenated end-to-end. This makes it easy for the user to load most of a shader from a file, but to have a standardized preamble that is prepended to some group of shaders.

The length​ can be either NULL or an array of count​ integers. These are the lengths of the corresponding strings in the string​ array. This allows you to use non-NULL-terminated strings. If you pass NULL, then OpenGL will assume all of the strings are NULL-terminated and will therefore compute the length in the usual way.

Once shader strings have been set into a shader object, it can be compiled with this function:

void glCompileShader(GLuint shader​);

It compiles the given shader.

Shader error handling

Compilation may or may not succeed. Shader compilation failure is not an OpenGL Error; you need to check for it specifically. This is done with a particular call to glGetShaderiv:

GLint success = 0;
glGetShaderiv(shader, GL_COMPILE_STATUS, &success);

If success​ is GL_FALSE, then the most recent compilation failed. Otherwise, it succeeded.

Shader compilation is pass/fail, but it is often useful to know why. This, like in most languages, is provided as text messages. OpenGL allows you to query a log containing this information. First, you must use glGetShaderiv to query the log's length:

GLint logSize = 0;
glGetShaderiv(shader, GL_INFO_LOG_LENGTH​, &logSize);

This tells you how many bytes to allocate; the length includes the NULL terminator. Once you have the length and have allocated sufficient memory, you can use this function to get the log:

void glGetShaderInfoLog(GLuint shader​, GLsizei maxLength​, GLsizei *length​, GLchar *infoLog​);

maxLength​ is the size of infoLog​; this tells OpenGL how many bytes at maximum it will write into infoLog​. length​ is a return value, specifying how many bytes it actually wrote into infoLog​; you may pass NULL if you don't care.

Shader compilation error checking.

GLuint shader = glCreateShader(...);

// Get strings for glShaderSource.
glShaderSource(shader, ...);

glCompileShader(shader);

GLint isCompiled = 0;
glGetShaderiv(shader, GL_COMPILE_STATUS, &isCompiled);
if(isCompiled == GL_FALSE)
{
	GLint maxLength = 0;
	glGetShaderiv(shader, GL_INFO_LOG_LENGTH, &maxLength);

	// The maxLength includes the NULL character
	std::vector<GLchar> errorLog(maxLength);
	glGetShaderInfoLog(shader, maxLength, &maxLength, &errorLog[0]);

	// Provide the infolog in whatever manor you deem best.
	// Exit with failure.
	glDeleteShader(shader); // Don't leak the shader.
	return;
}

// Shader compilation is successful.

Program setup

One you have successfully compiled the shader objects of interest, you can link them into a program. This begins by creating a program object via this command:

GLuint glCreateProgram();

The function takes no parameters.

After creating a program, the shader objects you wish to link to it must be attached to the program. This is done via this function:

void glAttachShader(GLuint program​, GLuint shader​);

This can be called multiple times with different shader​ objects.

Note: It is possible to attach multiple shader objects for the same shader stage to a program. This is perfectly legal. When linking them, all of the code will be combined. However, only one of those shader objects can have a main function. Linking like this will work very much like linking multiple object files in a C/C++ program: one .cpp file can call code in another .cpp file if it has forward declarations of functions. So you can have "libraries" of functions that individual shaders can choose to use.

Recommendation: That being said, while this power is available, it is best not to use it. It usually works, but because most OpenGL applications don't do this, it doesn't get as thoroughly tested as other parts of the OpenGL API. So you're likely to run into more driver bugs this way. Generally stick to having one shader object per shader stage.

Before linking

A number of parameters can be set up that will affect the linking process. This generally involves interfaces with the program. These include:

• Vertex shader input attribute locations.
• Fragment shader output color numbers.
• Transform feedback output capturing.
• Program separation.

You cannot change these values after linking; if you don't set them before linking, you can't set them at all.

Program linking

Linking can fail for many reasons, including but not limited to:

• Invalid matching between two shader stages in this program.
• Violation of various shader stage limitations. Some of these can be caught at compile-time, but others must wait until link time.
• Two or more global definitions of certain types have the same name in different shader stages, but different definitions.
• References to declared functions that are not defined.

Program link failure can be detected and responded to, in a similar way to shader compilation failure.

Once the program has been successfully linked, it can be used.

Linking and variables

Normally, shader objects for different shader stages don't interact. Each shader stage's code is separate from others. They have their own global variables, their own functions, etc.

This is not the case entirely. Certain definitions are considered shared between shader stages. Specifically, these include uniforms, buffer variables, and buffer-backed interface blocks.

If one of these is defined in one stage, another stage can define the same object with the same name and the exact same definition. If this happens, then there will only be one uniform/buffer variable/interface block visible from the introspection API. So shader stages in the same program can share uniform variables, allowing the same value to be set into both stages with one glUniform call.

For this to work however, the definitions must be exactly the same. This includes the order of the members, any user-defined data structures they use, array counts, everything. If two definitions in different stages have the same name, but different definitions, then there will be a linker error.

Cleanup

After linking (whether successfully or not), it is a good idea to detach all shader objects from the program. This is done via this function:

 void glDetachShader(GLuint program​, GLuint shader​);

shader​ must have been previously attached to program​.

If you do not intend to use this particular shader object in the linking of another program, you may delete it. This is done via glDeleteShader. Note that the deletion of a shader is deferred until the shader object is no longer attached to a program. Therefore, it is a good idea to detach shaders after linking.

Example

Full compile/link of a Vertex and Fragment Shader.

// Read our shaders into the appropriate buffers
std::string vertexSource = // Get source code for vertex shader.
std::string fragmentSource = // Get source code for fragment shader.

// Create an empty vertex shader handle
GLuint vertexShader = glCreateShader(GL_VERTEX_SHADER);

// Send the vertex shader source code to GL
// Note that std::string's .c_str is NULL character terminated.
const GLchar *source = (const GLchar *)vertexSource.c_str();
glShaderSource(vertexShader, 1, &source, 0);

// Compile the vertex shader
glCompileShader(vertexShader);

GLint isCompiled = 0;
glGetShaderiv(vertexShader, GL_COMPILE_STATUS, &isCompiled);
if(isCompiled == GL_FALSE)
{
	GLint maxLength = 0;
	glGetShaderiv(vertexShader, GL_INFO_LOG_LENGTH, &maxLength);

	// The maxLength includes the NULL character
	std::vector<GLchar> infoLog(maxLength);
	glGetShaderInfoLog(vertexShader, maxLength, &maxLength, &infoLog[0]);
	
	// We don't need the shader anymore.
	glDeleteShader(vertexShader);

	// Use the infoLog as you see fit.
	
	// In this simple program, we'll just leave
	return;
}

// Create an empty fragment shader handle
GLuint fragmentShader = glCreateShader(GL_FRAGMENT_SHADER);

// Send the fragment shader source code to GL
// Note that std::string's .c_str is NULL character terminated.
source = (const GLchar *)fragmentSource.c_str();
glShaderSource(fragmentShader, 1, &source, 0);

// Compile the fragment shader
glCompileShader(fragmentShader);

glGetShaderiv(fragmentShader, GL_COMPILE_STATUS, &isCompiled);
if (isCompiled == GL_FALSE)
{
	GLint maxLength = 0;
	glGetShaderiv(fragmentShader, GL_INFO_LOG_LENGTH, &maxLength);

	// The maxLength includes the NULL character
	std::vector<GLchar> infoLog(maxLength);
	glGetShaderInfoLog(fragmentShader, maxLength, &maxLength, &infoLog[0]);
	
	// We don't need the shader anymore.
	glDeleteShader(fragmentShader);
	// Either of them. Don't leak shaders.
	glDeleteShader(vertexShader);

	// Use the infoLog as you see fit.
	
	// In this simple program, we'll just leave
	return;
}

// Vertex and fragment shaders are successfully compiled.
// Now time to link them together into a program.
// Get a program object.
GLuint program = glCreateProgram();

// Attach our shaders to our program
glAttachShader(program, vertexShader);
glAttachShader(program, fragmentShader);

// Link our program
glLinkProgram(program);

// Note the different functions here: glGetProgram* instead of glGetShader*.
GLint isLinked = 0;
glGetProgramiv(program, GL_LINK_STATUS, (int *)&isLinked);
if (isLinked == GL_FALSE)
{
	GLint maxLength = 0;
	glGetProgramiv(program, GL_INFO_LOG_LENGTH, &maxLength);

	// The maxLength includes the NULL character
	std::vector<GLchar> infoLog(maxLength);
	glGetProgramInfoLog(program, maxLength, &maxLength, &infoLog[0]);
	
	// We don't need the program anymore.
	glDeleteProgram(program);
	// Don't leak shaders either.
	glDeleteShader(vertexShader);
	glDeleteShader(fragmentShader);

	// Use the infoLog as you see fit.
	
	// In this simple program, we'll just leave
	return;
}

// Always detach shaders after a successful link.
glDetachShader(program, vertexShader);
glDetachShader(program, fragmentShader);

Separate programs

Separate Shader Stages

Core in version		4.6
Core since version		4.1
Core ARB extension	ARB_separate_shader_objects

A program object can contain the code for multiple shader stages. The glUseProgram function only takes a single program, so you can only use a single program at a time for rendering. Therefore, you cannot mix-and-match code for different shader stages dynamically post-linking. Shader objects are not programs; they only hold compiled fragments of code, not fully useful programs.

There is a way to do this. This involves two alterations to the model presented above. The first is how the program is created; the second is in how it gets used.

To allow the use of multiple programs, were each program only provides some of the shader stage code, we must first create our programs specially. To signal that a program object is intended to be used with this separate program model, we must set a parameter on the program before linking. This is done as follows:

glProgramParameter(program, GL_PROGRAM_SEPARABLE, GL_TRUE);

Note: Again, this must done be before the program is linked.

There is an alternative method for creating separable programs. This represents a common use case: creating a program from a single set of shader source which provides the code for a single shader stage. The function to do this is:

GLuint glCreateShaderProgramv(GLenum type​, GLsizei count​, const char **strings​);

This works exactly as if you took count​ and strings​ strings, created a shader object from them of the type​ shader type, and then linked that shader object into a program with the GL_PROGRAM_SEPARABLE parameter. And then detaching and deleting the shader object.

This process can fail, just as compilation or linking can fail. The program infolog can thus contain compile errors as well as linking errors.

Note: glCreateShaderProgramv is a very useful tool, but it does have one major drawback. You cannot set any pre-link parameters into the resulting program. Most pre-link parameters can be set in the shader, but the transform feedback settings cannot. So if you need separable programs and transform feedback, you will have to use the two-stage process.

Separable programs are allowed to have shaders from more than one stage linked into them. While it is best to only use shaders from one stage (since the general point of using separable programs is the ability to mix-and-match freely), you do not have to.

Program pipelines

Creating a separable program is just the first step. The other thing you must do is change how the program is used.

To use multiple separable programs, they must first be assembled into an OpenGL Object type called a program pipeline. Unlike program or shader objects, these follow the standard OpenGL Object mode. Therefore, there is a glGenProgramPipelines function to create new pipeline names, a glDeleteProgramPipelines to delete them, and a glBindProgramPipeline to bind it to the context. Program pipeline objects do not have targets, so the last function only takes the pipeline to be bound.

Similar to Sampler Objects, program pipeline objects should only be bound when you intend to render with them (or set uniforms through them, as described below). The only state in program pipeline objects are the list of programs that contain the code for the various shader stages. This state is set by this function:

void glUseProgramStages(GLuint pipeline​, GLbitfield stages​, GLuint program​);

The given pipeline​ will get the shader code for the shader stages defined by the bitfield stages​ from the given program​. The stages​ bitfield determines which shader stages in program​ will provide code for those shader stages in the pipeline. These bits can be a combination of GL_VERTEX_SHADER_BIT, GL_TESS_CONTROL_SHADER_BIT, GL_TESS_EVALUATION_SHADER_BIT, GL_GEOMETRY_SHADER_BIT, GL_FRAGMENT_SHADER_BIT and GL_COMPUTE_SHADER_BIT. The bitfield can also be GL_ALL_SHADER_BITS, which is equivalent to all of the above. If the program​ has an active code for each stage mentioned in stages​, then that code will be used by the pipeline. If program​ is 0, then the given stages are cleared from the pipeline.

program​ must either be 0 or a separable program.

Rendering

Once you have a functioning program pipeline with all of the separate stages you would like to use, you can render with it. To do that, you must first bind the program pipeline with glBindProgramPipeline.

Warning: glUseProgram overrides glBindProgramPipeline. That is, if you have a program in use and a program pipeline bound, all rendering will use the program that is in use, not the pipeline programs. So make sure that glUseProgram(0) has been called.

After binding a pipeline, you can then render with those stages as normal, or dispatch compute work. Program pipelines can also be validated.

Uniforms and pipelines

Recommendation: The following explains a highly convoluted way to use glBindProgramPipeline to allow the user to set uniform state for one of the programs within a pipeline. You are strongly advised to ignore this section. Instead, use glProgramUniform to set uniforms directly into programs without having to bind or use the program. Only use this if you absolutely must remain backwards compatible with some functions that set uniforms on a program that is currently in use.

Binary upload

Program Binary

Core in version		4.6
Core since version		4.1
Core ARB extension	ARB_get_program_binary

Compiling and linking shaders, regardless of which method you use, can take a long time. The more shaders you have, the longer this process takes. It is often useful to be able to cache the result of program linking, so that this cached program can be reloaded much faster.

This is done via a set of calls. Given a successfully linked program, the user can fetch a block of binary data, in a certain format, that represents this program. The first step of this process is to get the length of this data by calling glGetProgram with GL_PROGRAM_BINARY_LENGTH on the program. Armed with this length, the actual binary can be obtained with this function:

void glGetProgramBinary(GLuint program​, GLsizei bufsize​, GLsizei *length​, GLenum *binaryFormat​, void *binary​);

bufsize​ is the maximum number of bytes available in binary​. length​ is an output value that states how many bytes were copied into binary​; it is optional and may be NULL. binaryFormat​ is an output value that specifies the format of the binary data. It is not optional, and it must be stored alongside the actual binary data.

Given the format and the binary data, a new program object can be created with this binary data. This is done via this function:

void glProgramBinary(GLuint program​, GLenum binaryFormat​, const void *binary​, GLsizei length​);

This function will upload the binary​ data (who's length​ is as given), which is in the given binaryFormat​, into the program​. If the upload is successful, then program​ has effectively had a successful link call performed on it.

This function can fail if binaryFormat​ is not a supported format. You can query the allowed formats with glGetIntegerv, using GL_NUM_PROGRAM_BINARY_FORMATS to get the count, and GL_PROGRAM_BINARY_FORMATS to get the formats. It can also fail for other reasons; you cannot guarantee that a binary can be loaded.

State of the program

Program objects contain certain state. The program binary only encapsulates the state of the program at the moment linking was successful. This means that all uniforms are reset to their default values (either specified in-shader or 0). Vertex attributes and fragment shader outputs will have the values assigned, as well as transform feedback data, interface block bindings, and so forth.

If glProgramBinary is successful, it should result in a program object that is identical to the original program object as it was immediately after linking.

If the original program was separable, then the program built from the binary will also be separable. And vice-versa.

Binary limitations

Program binary formats are not intended to be transmitted. It is not reasonable to expect different hardware vendors to accept the same binary formats. It is not reasonable to expect different hardware from the same vendor to accept the same binary formats.

Indeed, you cannot expect the cached version to work even on the same machine. Driver updates between when the data was cached and when it is reloaded can change the acceptable binary formats. Therefore, glProgramBinary can fail frequently. If you use this functionality, you must have a fallback for creating your shaders if the binary is rejected.

Validation

A program object, or program pipeline object, must be valid to be used in rendering operations. As much of this validity is checked at link-time as possible; however, some of it references the current OpenGL state. Therefore, some of it must be tested at runtime. For program pipelines, some validity that would have been checked at link-time for non-separable programs (such as interface matching) must be checked at runtime.

The validity of a program or pipeline object can be checked at any time using these functions:

Here are the rules of program object validation:

Pipeline validation

Pipeline object validation must also check the following: