Shader Compilation

From OpenGL Wiki
Jump to navigation Jump to search

Shader Compilation is the process of text in the OpenGL Shading Language and loading it into OpenGL to be used as a Shader. OpenGL has three ways to compile shader text into useable OpenGL objects. All of these forms of compilation produce a Program Object.

Shader and program objects

A Program Object can contain the executable code for all of the Shader stages, such that all that is needed to render is to bind one program object. Building programs that contain multiple shader stages requires a two-stage compilation process.

This two-stage compilation process mirrors the standard compile/link setup for C and C++ source code. C/C++ text is first fed through a compiler, thus producing an object file. To get the executable code, one or more object files must be linked together.

With this method of program creation, shader text is first fed through a compiler, thus producing a shader object. To get the executable program object, one or more shader objects must be linked together.

Shader object compilation

The first step is to create shader objects for each shader that you intend to use and compile them. To create a shader object, you call this function:

GLuint glCreateShader(GLenum shaderType​);

This creates an empty shader object for the shader stage given by given shaderType​. The shader type must be one of GL_VERTEX_SHADER, GL_TESS_CONTROL_SHADER, GL_TESS_EVALUATION_SHADER, GL_GEOMETRY_SHADER, GL_FRAGMENT_SHADER, or GL_COMPUTE_SHADER. Note that the control and evaluation shaders require GL 4.0 (or ARB_tessellation_shader), and the compute shader requires GL 4.3 (or ARB_compute_shader).

Once you have a shader object, you will need to give it the actual text string representing the GLSL source code. That is done via this function:

void glShaderSource(GLuint shader​, GLsizei count​, const GLchar **string​, const GLint *length​);

This function takes the array of strings, given by string​ and stores it into shader​. Any previously stored strings are removed. count​ is the number of individual strings. OpenGL will copy these strings into internal memory.

When the shader is compiled, it will be compiled as if all of the given strings were concatenated end-to-end. This makes it easy for the user to load most of a shader from a file, but to have a standardized preamble that is prepended to some group of shaders.

The length​ can be either NULL or an array of count​ integers. These are the lengths of the corresponding strings in the string​ array. This allows you to use non-NULL-terminated strings. If you pass NULL, then OpenGL will assume all of the strings are NULL-terminated and will therefore compute the length in the usual way.

Once shader strings have been set into a shader object, it can be compiled with this function:

void glCompileShader(GLuint shader​);

It compiles the given shader.

Shader error handling

Compilation may or may not succeed. Shader compilation failure is not an OpenGL Error; you need to check for it specifically. This is done with a particular call to glGetShaderiv:

GLint success = 0;
glGetShaderiv(shader, GL_COMPILE_STATUS, &success);

If success​ is GL_FALSE, then the most recent compilation failed. Otherwise, it succeeded.

Shader compilation is pass/fail, but it is often useful to know why. This, like in most languages, is provided as text messages. OpenGL allows you to query a log containing this information. First, you must use glGetShaderiv to query the log's length:

GLint logSize = 0;
glGetShaderiv(shader, GL_INFO_LOG_LENGTH, &logSize);

This tells you how many bytes to allocate; the length includes the NULL terminator. Once you have the length and have allocated sufficient memory, you can use this function to get the log:

void glGetShaderInfoLog(GLuint shader​, GLsizei maxLength​, GLsizei *length​, GLchar *infoLog​);

maxLength​ is the size of infoLog​; this tells OpenGL how many bytes at maximum it will write into infoLog​. length​ is a return value, specifying how many bytes it actually wrote into infoLog​; you may pass NULL if you don't care.

Shader compilation error checking.

GLuint shader = glCreateShader(...);

// Get strings for glShaderSource.
glShaderSource(shader, ...);

glCompileShader(shader);

GLint isCompiled = 0;
glGetShaderiv(shader, GL_COMPILE_STATUS, &isCompiled);
if(isCompiled == GL_FALSE)
{
	GLint maxLength = 0;
	glGetShaderiv(shader, GL_INFO_LOG_LENGTH, &maxLength);

	// The maxLength includes the NULL character
	std::vector<GLchar> errorLog(maxLength);
	glGetShaderInfoLog(shader, maxLength, &maxLength, &errorLog[0]);

	// Provide the infolog in whatever manor you deem best.
	// Exit with failure.
	glDeleteShader(shader); // Don't leak the shader.
	return;
}

// Shader compilation is successful.

Program setup

One you have successfully compiled the shader objects of interest, you can link them into a program. This begins by creating a program object via this command:

GLuint glCreateProgram();

The function takes no parameters.

After creating a program, the shader objects you wish to link to it must be attached to the program. This is done via this function:

void glAttachShader(GLuint program​, GLuint shader​);

This can be called multiple times with different shader​ objects.

Note: It is possible to attach multiple shader objects for the same shader stage to a program. This is perfectly legal. When linking them, all of the code will be combined. However, only one of those shader objects can have a main function. Linking like this will work very much like linking multiple object files in a C/C++ program: one .cpp file can call code in another .cpp file if it has forward declarations of functions. So you can have "libraries" of functions that individual shaders can choose to use.
Recommendation: That being said, while this power is available, it is best not to use it. It usually works, but because most OpenGL applications don't do this, it doesn't get as thoroughly tested as other parts of the OpenGL API. So you're likely to run into more driver bugs this way. Generally stick to having one shader object per shader stage.

Before linking

A number of parameters can be set up that will affect the linking process. This generally involves interfaces with the program. These include:

You cannot change these values after linking; if you don't set them before linking, you can't set them at all.

Program linking

Linking can fail for many reasons, including but not limited to:

Program link failure can be detected and responded to, in a similar way to shader compilation failure.

Once the program has been successfully linked, it can be used.

Linking and variables

Normally, shader objects for different shader stages don't interact. Each shader stage's code is separate from others. They have their own global variables, their own functions, etc.

This is not the case entirely. Certain definitions are considered shared between shader stages. Specifically, these include uniforms, buffer variables, and buffer-backed interface blocks.

If one of these is defined in one stage, another stage can define the same object with the same name and the exact same definition. If this happens, then there will only be one uniform/buffer variable/interface block visible from the introspection API. So shader stages in the same program can share uniform variables, allowing the same value to be set into both stages with one glUniform call.

For this to work however, the definitions must be exactly the same. This includes the order of the members, any user-defined data structures they use, array counts, everything. If two definitions in different stages have the same name, but different definitions, then there will be a linker error.

Cleanup

After linking (whether successfully or not), it is a good idea to detach all shader objects from the program. This is done via this function:

 void glDetachShader(GLuint program​, GLuint shader​);

shader​ must have been previously attached to program​.

If you do not intend to use this particular shader object in the linking of another program, you may delete it. This is done via glDeleteShader. Note that the deletion of a shader is deferred until the shader object is no longer attached to a program. Therefore, it is a good idea to detach shaders after linking.

Example

Full compile/link of a Vertex and Fragment Shader.

// Read our shaders into the appropriate buffers
std::string vertexSource = // Get source code for vertex shader.
std::string fragmentSource = // Get source code for fragment shader.

// Create an empty vertex shader handle
GLuint vertexShader = glCreateShader(GL_VERTEX_SHADER);

// Send the vertex shader source code to GL
// Note that std::string's .c_str is NULL character terminated.
const GLchar *source = (const GLchar *)vertexSource.c_str();
glShaderSource(vertexShader, 1, &source, 0);

// Compile the vertex shader
glCompileShader(vertexShader);

GLint isCompiled = 0;
glGetShaderiv(vertexShader, GL_COMPILE_STATUS, &isCompiled);
if(isCompiled == GL_FALSE)
{
	GLint maxLength = 0;
	glGetShaderiv(vertexShader, GL_INFO_LOG_LENGTH, &maxLength);

	// The maxLength includes the NULL character
	std::vector<GLchar> infoLog(maxLength);
	glGetShaderInfoLog(vertexShader, maxLength, &maxLength, &infoLog[0]);
	
	// We don't need the shader anymore.
	glDeleteShader(vertexShader);

	// Use the infoLog as you see fit.
	
	// In this simple program, we'll just leave
	return;
}

// Create an empty fragment shader handle
GLuint fragmentShader = glCreateShader(GL_FRAGMENT_SHADER);

// Send the fragment shader source code to GL
// Note that std::string's .c_str is NULL character terminated.
source = (const GLchar *)fragmentSource.c_str();
glShaderSource(fragmentShader, 1, &source, 0);

// Compile the fragment shader
glCompileShader(fragmentShader);

glGetShaderiv(fragmentShader, GL_COMPILE_STATUS, &isCompiled);
if (isCompiled == GL_FALSE)
{
	GLint maxLength = 0;
	glGetShaderiv(fragmentShader, GL_INFO_LOG_LENGTH, &maxLength);

	// The maxLength includes the NULL character
	std::vector<GLchar> infoLog(maxLength);
	glGetShaderInfoLog(fragmentShader, maxLength, &maxLength, &infoLog[0]);
	
	// We don't need the shader anymore.
	glDeleteShader(fragmentShader);
	// Either of them. Don't leak shaders.
	glDeleteShader(vertexShader);

	// Use the infoLog as you see fit.
	
	// In this simple program, we'll just leave
	return;
}

// Vertex and fragment shaders are successfully compiled.
// Now time to link them together into a program.
// Get a program object.
GLuint program = glCreateProgram();

// Attach our shaders to our program
glAttachShader(program, vertexShader);
glAttachShader(program, fragmentShader);

// Link our program
glLinkProgram(program);

// Note the different functions here: glGetProgram* instead of glGetShader*.
GLint isLinked = 0;
glGetProgramiv(program, GL_LINK_STATUS, (int *)&isLinked);
if (isLinked == GL_FALSE)
{
	GLint maxLength = 0;
	glGetProgramiv(program, GL_INFO_LOG_LENGTH, &maxLength);

	// The maxLength includes the NULL character
	std::vector<GLchar> infoLog(maxLength);
	glGetProgramInfoLog(program, maxLength, &maxLength, &infoLog[0]);
	
	// We don't need the program anymore.
	glDeleteProgram(program);
	// Don't leak shaders either.
	glDeleteShader(vertexShader);
	glDeleteShader(fragmentShader);

	// Use the infoLog as you see fit.
	
	// In this simple program, we'll just leave
	return;
}

// Always detach shaders after a successful link.
glDetachShader(program, vertexShader);
glDetachShader(program, fragmentShader);

Separate programs

Separate Shader Stages
Core in version 4.6
Core since version 4.1
Core ARB extension ARB_separate_shader_objects

A program object can contain the code for multiple shader stages. The glUseProgram function only takes a single program, so you can only use a single program at a time for rendering. Therefore, you cannot mix-and-match code for different shader stages dynamically post-linking. Shader objects are not programs; they only hold compiled fragments of code, not fully useful programs.

There is a way to do this. This involves two alterations to the model presented above. The first is how the program is created; the second is in how it gets used.

To allow the use of multiple programs, were each program only provides some of the shader stage code, we must first create our programs specially. To signal that a program object is intended to be used with this separate program model, we must set a parameter on the program before linking. This is done as follows:

glProgramParameter(program, GL_PROGRAM_SEPARABLE, GL_TRUE);
Note: Again, this must done be before the program is linked.

There is an alternative method for creating separable programs. This represents a common use case: creating a program from a single set of shader source which provides the code for a single shader stage. The function to do this is:

GLuint glCreateShaderProgramv(GLenum type​, GLsizei count​, const char **strings​);

This works exactly as if you took count​ and strings​ strings, created a shader object from them of the type​ shader type, and then linked that shader object into a program with the GL_PROGRAM_SEPARABLE parameter. And then detaching and deleting the shader object.

This process can fail, just as compilation or linking can fail. The program infolog can thus contain compile errors as well as linking errors.

Note: glCreateShaderProgramv is a very useful tool, but it does have one major drawback. You cannot set any pre-link parameters into the resulting program. Most pre-link parameters can be set in the shader, but the transform feedback settings cannot. So if you need separable programs and transform feedback, you will have to use the two-stage process.

Program pipelines

Creating a separable program is just the first step. The other thing you must do is change how the program is used.

Separable programs must be assembled into an OpenGL Object type called a program pipeline.

Binary upload

Program Binary
Core in version 4.6
Core since version 4.1
Core ARB extension ARB_get_program_binary


Error handling

Interface matching