Difference between revisions of "Multisampling"

From OpenGL Wiki
Jump to navigation Jump to search
(Better description of what multisampling is.)
Line 1: Line 1:
Multisampling, also known as multisample antialiasing (MSAA), is one method for achieving full-screen antialiasing (FSAA). With multisampling, each pixel at the edge of a polygon is sampled multiple times. For each sample-pass, a slight offset is applied to all screen coordinates. This offset is smaller than the actual size of the pixels. By averaging all these samples, the result is a smoother transition of the colors at the edges. Unlike supersampling (SSAA) which can result in the same pixel being shaded multiple times per pixel, multisampling runs the fragment program just once per pixel rasterized.  However with MSAA multiple depth/stencil comparisons are performed ''per sample'', one for each of the subsamples, which gives you sub-pixel spatial precision on your geometry and nice, smoothed edges on your polygons.
+
'''Multisampling''' is a process for reducing aliasing at the edges of rasterized primitives.
  
== History ==
+
== Aliasing ==
Before GL_ARB_multisample extension, the edges of lines, polygons, and points could be selectively antialiased using using glEnable(GL_LINE_SMOOTH), glEnable(GL_POLYGON_SMOOTH), glEnable(GL_POINT_SMOOTH) respectively, combined with a blending function, such as glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA). Such features were typically not implemented in hardware in early consumer graphics cards at the time, and were done in software resulting in poor performance. More expensive "workstation" graphics cards from HP, Sun, and SGI at the time did implement these features in hardware. Modern programs should not make use of these features.
 
  
== Rendering with Multisampling ==
+
[https://en.wikipedia.org/wiki/Aliasing Aliasing] is a signal processing term caused by analog-to-digital conversions. When an analog signal is converted to a digital signal, this is done by reading the value of the analog signal at a variety of discrete locations called samples. If not enough samples are used, the digital equivalent of the analog signal can manifest unusual patterns. This happens because there are multiple analog signals that could have produced the digital signal using the same sampling pattern. Because the digital signal is not a unique representation of the analog signal, the effects that this produces are called "aliasing" (one signal acts as an "alias" for another).
  
There are two pieces to rendering with multisampling:
+
In terms of computer graphics, 2D rendering is a form of analog-to-digital conversion. The world of primitives is an analog world described mathematically. Rendering it to a series of discrete pixels creates a digital signal representing that world. If not enough samples are used in creating those discrete pixels, the resulting digital image can exhibit aliasing effects.
  
# Allocating a multisample render target (window or [[FBO]]), and
+
Aliasing effects in graphics tend to appear as jagged edges of triangles or textures that look extremely rough in areas that should appear smooth, particularly edge on. When animation gets involved, aliasing becomes extremely noticeable, as aliasing pixels tend to shift from color to color abruptly instead of smoothly.
# Enabling multisample rasterization (i.e. <code>glEnable( GL_MULTISAMPLE )</code>)
 
  
While the second step is standardized across all render target types and platforms, the first is only standardized for [[FBO]]s (as it is totally internal to GL).  When rendering to a window, the allocation method depends on the platform-specific GL integration layer in-use (e.g. WGL, GLX, AGL, etc.).  GLUT provides a wrapper around some of these so you don't have to care.
+
== Antialiasing ==
  
The following sections describe how to perform MSAA render target allocation (#1) for various render target types and platforms.
+
Combating the visual effects of aliasing is simple, so long as performance is irrelevant. Aliasing is the effect of not using enough samples in analog-to-digital conversions. Thus, all true antialiasing techniques revolve around increasing the number of samples used.
  
== Allocating a Multisample Render Target ==
+
[[Texture Filtering|Texture filtering]] is a form of antialising, applied specifically to aliasing caused by accessing textures. Linear filtering mixes together neighboring samples instead of just using one. Mipmaps of a texture are essentially ways of pre-computing an approximation of accessing a large area of a texture. Each texel in a mipmap represents the average of several texels from the higher mipmap. [[Anisotropic Filtering|Anisotropic filtering's]] ability to access different locations from a texture is also a form of antialiasing, fetching multiple samples to compute a more reasonable value.
  
=== Render-to-FBO ===
+
But texture filtering only deals with aliasing that results from accessing textures and computations based on such accesses. Aliasing at the edge of primitives is not affected by such filtering.
  
As mentioned, allocation of multisample off-screen [[FBO]]s is platform independent. Here's an example:
+
A more general form of antialiasing is the simplest: render at a higher resolution, then compute the final image by averaging the values from the higher resolution image that correspond to each pixel. This is commonly called "supersampling".
  
<source lang="c">
+
In supersampling, each pixel in the eventual destination image gets its data from multiple pixels in the higher resolution image. The high-res pixels that correspond to a particular destination pixel are called "samples". Given this idea, we can think about a supersampled image as having the same pixel resolution as the destination, but with each pixel storing multiple samples of data.
  glGenTextures( 1, &tex );
 
  glBindTexture( GL_TEXTURE_2D_MULTISAMPLE, tex );
 
  glTexImage2DMultisample( GL_TEXTURE_2D_MULTISAMPLE, num_samples, GL_RGBA8, width, height, false );
 
  
  glGenFramebuffers( 1, &fbo );
+
An image where each pixel stores multiple samples is a "multisampled image".
  glBindFramebuffer( GL_FRAMEBUFFER, fbo );
 
  glFramebufferTexture2D( GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D_MULTISAMPLE, tex, 0 );
 
  
  GLenum status = glCheckFramebufferStatus( target );
+
When we do rasterization with supersampling, the primitive is broken down into multiple samples for each pixel. Each sample is taken at a different location within the pixel's area. So each sample contains all of the rasterization products and everything following them in the [[Rendering Pipeline Overview|pipeline]]. So for each sample in the multisampled image, we must produce a [[Fragment]], execute a [[Fragment Shader]] on it to compute colors, do a bunch of [[Per-Sample Processing|other operations]], and write the sample.
  ...
 
</source>
 
  
Once you've rendered your scene into the multisampled FBO, you can display the anti-aliased result on the screen by blitting your multisampled FBO to the back buffer using <code>glBlitFramebuffer()</code>. Before doing so, you must first remember to specify the back buffer as your draw buffer, and to specify your multisampled FBO as the read frame buffer, like so:
+
As previously stated, each sample within a multisampled pixel comes from a specific location within the area of that pixel. When we attempt to rasterize a primitive for a pixel, we sample the primitive at all of the sample locations within that pixel. If any of those locations fall outside of the primitive's area (because the pixel is at the edge of the primitive), then the samples outside of the area will not generate fragments.
  
<source lang="c">
+
So in every way, supersampling renders to a higher-resolution image; it's just easier to talk about it in terms of adding samples within a pixel.
  glBindFramebuffer(GL_DRAW_FRAMEBUFFER, 0);   // Make sure no FBO is set as the draw framebuffer
 
  glBindFramebuffer(GL_READ_FRAMEBUFFER, fbo); // Make sure your multisampled FBO is the read framebuffer
 
  glDrawBuffer(GL_BACK);                      // Set the back buffer as the draw buffer
 
  glBlitFramebuffer(0, 0, width, height, 0, 0, width, height, GL_COLOR_BUFFER_BIT, GL_NEAREST);
 
</source>
 
  
Doing this blit will automatically ''resolve'' the multisampled FBO. In other words, for each texel in the multisampled FBO, the blit will blend together the texel's samples, and this blended color is what is written into the corresponding pixel on the back buffer. If you wish, you can resolve your multisampled FBO into a different off-screen FBO by setting that FBO as your draw buffer.
+
While this is very simple and easy to implement, it's also obviously expensive. It has all of the downsides of rendering at high resolutions: lots of added rasterization, lots of shader executions, and those multisampled images consume lots of memory and therefore bandwidth. Plus, to compute the final image, we have to take time to average the colors from the multisampled image into its final resolution.
  
=== Render-to-Window ===
+
== Multisampling ==
  
==== Microsoft Windows (using WGL) ====
+
Reasonable use of texture filtering can reduce aliasing within the area of a primitive. As such, supersampling's primary value is in dealing with aliasing at the edges of primitives. But the cost of supersampling affects all parts of rasterization, so the costs tend to outweigh the benefits.
  
The WGL functions required to create a multisampled OpenGL context are not available until a valid OpenGL context is made current to the thread. This leads to a [[Creating_an_OpenGL_Context#Proper_Context_Creation|temporary context]] code path which can get very involved with platform specific detailed. Users of libraries such as GLUT, GLEW, or GLee can significantly reduce the amount of effort required. The rest of this section assumes that valid context is present and all wgl extension function pointers have been obtained, if not, please see how to create a [[Creating_an_OpenGL_Context#Proper_Context_Creation|temporary context]].
+
Multisampling is a small modification of the supersampling algorithm that is more focused on edge antialiasing.
  
A valid pixel format for the framebuffer is choose using the wglChoosePixelFormatARB function with a list of attributes to specify the multisampling properties. In order to choose a framebuffer format that incorporates multisampling, you must add it to the list of attributes. For example, this list of attributes ''does not'' select a multisampled pixel format:
+
In multisampling, everything is set up exactly like supersampling. We still render to multisampled images. The rasterizer still generates (most of) its rasterization data for each sample. Blending, depth testing, and the like still happen per-sample.
  
<source lang="c">
+
The only change with multisampling is that the fragment shader is ''not'' executed per-sample. It is executed at a lower frequency than the number of samples per pixel. Exactly how frequently it gets executed depends on the hardware. Some hardware may maintain a 4:1 ratio, such that the FS is executed once for each 4 samples in the multisample rendering.
    int attributes[] = {
 
      WGL_DRAW_TO_WINDOW_ARB, GL_TRUE,
 
      WGL_SUPPORT_OPENGL_ARB, GL_TRUE,
 
      WGL_DOUBLE_BUFFER_ARB, GL_TRUE,
 
      WGL_PIXEL_TYPE_ARB, WGL_TYPE_RGBA_ARB,
 
      WGL_COLOR_BITS_ARB, 32,
 
      WGL_DEPTH_BITS_ARB, 24,
 
      WGL_STENCIL_BITS_ARB, 8,
 
      0
 
    };
 
</source>
 
  
To consider multisampled visuals, the WGL_SAMPLE_BUFFERS_ARB and WGL_SAMPLES_ARB attributes must be present. The WGL_SAMPLE_BUFFERS_ARB must be set to 1, and WGL_SAMPLES_ARB is the number of samples, e.g. for 8x multisampling, WGL_SAMPLES_ARB would be set to 8. This attribute list is the same as above, but considers 4x multisampled pixel formats too:
+
That brings up an interesting question. If the FS is executed at a lower frequency, how do the other samples get their post-FS values? That is, for the samples that ''don't'' correspond to a FS execution, from where do they get their fragment values?
  
<source lang="c">
+
The answer is simple: any FS invocations executed in a multisampled pixel will copy their values to multiple samples. So in our 4:1 example above, if the multisample image contains 4 samples per pixel, then those four samples will get the same fragment values.
    int attributes[] = {
 
      WGL_DRAW_TO_WINDOW_ARB, GL_TRUE,
 
      WGL_SUPPORT_OPENGL_ARB, GL_TRUE,
 
      WGL_DOUBLE_BUFFER_ARB, GL_TRUE,
 
      WGL_PIXEL_TYPE_ARB, WGL_TYPE_RGBA_ARB,
 
      WGL_COLOR_BITS_ARB, 32,
 
      WGL_DEPTH_BITS_ARB, 24,
 
      WGL_STENCIL_BITS_ARB, 8,
 
      WGL_SAMPLE_BUFFERS_ARB, 1, // Number of buffers (must be 1 at time of writing)
 
      WGL_SAMPLES_ARB, 4,       // Number of samples
 
      0
 
    };
 
</source>
 
  
Once the correct pixel format is found, creating a context proceeds as normal.
+
In essence, multisampling is supersampling where the sample rate of the fragment shader (and all of its attendant operations) is lower than the number of samples per pixel.
  
==== X Windows (using GLX) ====
+
Note however that the depth value is still computed per-''sample'', not per fragment. This means that, unless the fragment shader replaces the fragment's depth value, the depth value for different samples will have different values. And the sample values in the depth buffers can have different values as well. Depth and stencil tests happen per-sample, and the failure of individual samples prevent the fragment's values from being written to those samples.
  
This is the method you'd use on UNIX/Linux if you want to create an X window with multisampling capability.  There are a few methods for doing this, but the following locates an MSAA FBConfig, looks up its XVisual, and then creates an X window in that visual.  You'll notice some similarities with the WGL method above.
+
=== Coverage ===
  
<source lang="c">
+
There still remain a few details to cover. When discussing supersampling, it was mentioned that samples that are outside of the area of the primitive being rasterized don't get values. We can think of the samples in a fragment being rasterized as having a binary state of being covered or not covered. The set of samples in a fragment area covered by the primitive represents the "coverage" of that fragment.
  // --- Find a MSAA FBConfig ---
 
  static const int Visual_attribs[] =
 
  {
 
      GLX_X_RENDERABLE    , True,
 
      GLX_DRAWABLE_TYPE  , GLX_WINDOW_BIT,
 
      GLX_RENDER_TYPE    , GLX_RGBA_BIT,
 
      GLX_X_VISUAL_TYPE  , GLX_TRUE_COLOR,
 
      GLX_RED_SIZE        , 8,
 
      GLX_GREEN_SIZE      , 8,
 
      GLX_BLUE_SIZE      , 8,
 
      GLX_ALPHA_SIZE      , 8,
 
      GLX_DEPTH_SIZE      , 24,
 
      GLX_STENCIL_SIZE    , 8,
 
      GLX_DOUBLEBUFFER    , True,
 
      GLX_SAMPLE_BUFFERS  , 1,            // <-- MSAA
 
      GLX_SAMPLES        , 4,            // <-- MSAA
 
      None
 
  };
 
  
  int attribs [ 100 ] ;
+
When a depth or stencil test fails for a sample, this modifies the coverage for the fragment by turning off that particular sample for the fragment.
  memcpy( attribs, Visual_attribs, sizeof( Visual_attribs ) );
 
  
  GLXFBConfig fbconfig = 0;
+
The concept of coverage is important because it can be accessed and manipulated by a fragment shader.
  int        fbcount;
 
  GLXFBConfig *fbc = glXChooseFBConfig( display, screen, attribs, &fbcount );
 
  if ( fbc )
 
  {
 
    if ( fbcount >= 1 )
 
      fbconfig = fbc[0];
 
    XFree( fbc );
 
  }
 
  
  if ( !fbconfig )
+
In {{require|4.0|sample_shading}}, the fragment shader [[Fragment Shader#System inputs|has a bitmask input that represents the samples covered by that fragment]]: {{code|gl_SampleMaskIn}}. It also has an [[Fragment Shader#Other outputs|output variable that can be used to set the coverage of the fragment]]: {{code|gl_SampleMask}}. Note that the output sample mask will be ANDed with the physical coverage of the primitive, so you cannot use this to write to samples that are outside of the space actually covered by the primitive.
  {
 
    printf( "Failed to get MSAA GLXFBConfig\n" );
 
    exit(1);
 
  }
 
  
  // --- Get its VisualInfo ---
+
The alpha value of color number 0 (index 0) generated by the fragment shader can also be set to manipulate the coverage mask (this feature predates {{code|gl_SampleMask}} and is less capable and more hardware-dependent, so you should just modify the sample mask directly if you can). This is activated by using {{apifunc|glEnable|({{enum|GL_SAMPLE_ALPHA_TO_COVERAGE}})}}. When activated, the alpha value will manipulate the coverage mask, such that an alpha of 1.0 represents full coverage and a value of 0.0 represents no coverage. The details of the implementation of the mapping of alpha to coverage are hardware-specific, but the mapping is expected to be linear.
  XVisualInfo *visinfo = glXGetVisualFromFBConfig( display, fbconfig );
 
  if ( !visinfo )
 
  {
 
    printf( "Failed to get XVisualInfo\n" );
 
    exit(1);
 
  }
 
  printf( "X Visual ID = 0x%.2x\n", int( visinfo->visualid ) );
 
  
  // --- Now just create an X window in that visual ---
+
The goal of this tool is to make multisampling act somewhat like alpha blending, with values closer to 1 taking up more samples in an area than values closer to 0.
  XSetWindowAttributes winAttr ;
 
  
  winAttr.event_mask = StructureNotifyMask | KeyPressMask ;
+
Much like changing the coverage mask directly, alpha-to-coverage will never make the coverage mask affect samples that aren't physically covered by the primitive area.
  winAttr.background_pixmap = None ;
 
  winAttr.background_pixel  = 0    ;
 
  winAttr.border_pixel      = 0    ;
 
  
  winAttr.colormap = XCreateColormap( display, root_win,
+
Because this functionality uses the alpha value being output, that alpha value is not meaningful anymore. If you don't want to use [[Blending]] or [[Write Mask]]ing to avoid changes to the alpha, you can {{apifunc|glEnable|({{enum|GL_SAMPLE_ALPHA_TO_ONE}})}}. This causes the alpha for the fragment to be converted to 1.0 ''after'' the point where it is used to modify the coverage mask.
                                      visinfo->visual, AllocNone );
 
 
 
  unsigned int mask = CWBackPixmap | CWBorderPixel | CWColormap | CWEventMask;
 
  
  Window win = XCreateWindow ( display, root_win,
+
=== Primitive edges ===
                              WIN_XPOS, WIN_YPOS,
 
                              WIN_XRES, WIN_YRES, 0,
 
                              visinfo->depth, InputOutput,
 
                              visinfo->visual, mask, &winAttr ) ;
 
  
  XStoreName( display, win, "My GLX Window");
+
There is one major caveat to executing the fragment shader at a rate lower than the number of samples: the location of the FS being executed within the pixel. In multisampling, because the FS invocation will be broadcast to multiple samples, the location of that FS invocation within the pixel will not always match the location of the samples that receive that data.
</source>
 
  
==== macOS (using Cocoa) ====
+
But more important, it is entirely possible that the sample location used by the FS invocation could be ''outside'' of the primitive's area, if the pixel is at the edge of the primitive. This is fine for many FS computations, as interpolation of values past the edge of a primitive can still work mathematically speaking.
  
''TBD''
+
What may not be fine is what you *do* with that interpolated value. A particular FS execution may not function correctly in the event that the interpolated values represent a location outside of the primitive area. For example, you might apply a square root to an interpolated value, expecting that the value will never be negative. But interpolation outside of the primitive may force it to be negative.
  
==== Cross-platform (using GLUT) ====
+
This is what the [[Interpolation qualifier|{{code|centroid}} interpolation qualifier]] fixes. Any fragment shader inputs qualified by this identifier will be interpolated within the area of the primitive.
  
GLUT is a wrapper around the platform-specific GL integration layers (e.g. WGL, GLX, etc.) which allows you to prototype simple cross-platform GL examples quickly.  In GLUT, the key to allocating an MSAA window is this:
+
=== Multisample Resolve ===
  
<source lang="c">
+
Once a multisample image is built and rendered to, if it is to be viewed, it must be reduced down to a single-sample per-pixel. The process of downsampling a multisample image is called the "multisample resolve". In OpenGL, to perform a multisample resolve, you use a [[Framebuffer#Blitting|blit operation]] from a multisampled framebuffer to a single-sampled one. Note that such a resolve blit operation cannot also rescale the image or change its format.
  glutInitDisplayMode( ... | GLUT_MULTISAMPLE );
 
</source>
 
  
which directs GLUT to allocate an MSAA-capable window. The following is a complete example showing how to render with multisampling in a simple GLUT test program:
+
If you create a multisampled [[Default Framebuffer]], the back buffer is considered multisampled, but the front buffer is not. So swapping the buffers is equivalent to doing a multisample resolve operation.
  
<source lang="c">
+
=== Per-sample shading ===
#include <GL/gl.h>
 
#include <GL/glut.h>
 
#include <GL/glu.h>
 
  
#include <stdio.h>
+
{{infobox feature
 +
| name = Per-sample shading
 +
| core = 4.0
 +
| arb_extension = {{extref|sample_shading}}
 +
}}
  
float rotation_angle = 0;
+
The whole idea of multisampling is that the FS should execute at a lower frequency than the sample count. However, it is sometimes useful to ''force'' the fragment shader to execute at the sample rate. For example, if you are doing post-processing effects on a multisample image before resolving it, then you need to execute those effects on each sample within each pixel. And the results could be very different depending on overlapping primitives and the like.
int msaa = 1;
 
  
void display()
+
This is called "per-sample shading", and it effectively transforms multisampling into supersampling.
{
 
  int err = 0;
 
  glClear(GL_COLOR_BUFFER_BIT);
 
  
  glMatrixMode(GL_PROJECTION);
+
Per-sample shading is activated for a shader if you use the {{code|sample}} interpolation qualifier on any FS input value, or if you access {{code|gl_SamplePosition}} or {{code|gl_SampleID}} in the FS.
  glLoadIdentity();
 
  glOrtho(-1, 1, -1, 1, -1, 1);
 
  
  glMatrixMode(GL_MODELVIEW);
+
== Smooth antialiasing ==
  glLoadIdentity();
+
{{deprecated|section=}}
  
  glRotatef(rotation_angle, 0, 0, 1);
+
Before GL_ARB_multisample extension, the edges of lines, polygons, and points could be selectively antialiased using using {{apifunc|glEnable|(GL_LINE_SMOOTH)}}, {{code|glEnable(GL_POLYGON_SMOOTH)}}, {{code|glEnable(GL_POINT_SMOOTH)}} respectively, combined with a blending function, such as {{apifunc|glBlendFunc|(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA)}}. Such features were typically not implemented in hardware in early consumer graphics cards at the time, and were done in software resulting in poor performance. More expensive "workstation" graphics cards from HP, Sun, and SGI at the time did implement these features in hardware.
  
  glColor4f(1, 0, 0, 1);
 
 
  if (msaa)
 
  {
 
    glEnable(GL_MULTISAMPLE_ARB);
 
    printf("MSAA on\n");
 
  }
 
  else
 
  {
 
    printf("MSAA off\n");
 
    glDisable(GL_MULTISAMPLE_ARB);
 
  }
 
 
  glRectf(-.5, -.5, .5, .5);
 
 
 
  glutSwapBuffers();
 
  err = glGetError();
 
  if (err)
 
    fprintf(stderr, "%s\n", gluErrorString(err));
 
}
 
 
void mouse(int button, int state, int x, int y)
 
{
 
  if (state == GLUT_DOWN)
 
  {
 
    msaa = !msaa;
 
    glutPostRedisplay();
 
  }
 
}
 
 
void reshape(int width, int height)
 
{
 
  glViewport(0, 0, width, height);
 
}
 
 
int
 
main (int argc, char** argv)
 
{
 
  glutInit(&argc, argv);
 
  glutInitDisplayMode( GLUT_RGBA | GLUT_DOUBLE | GLUT_MULTISAMPLE);
 
  glutCreateWindow(argv[0]);
 
 
  glutDisplayFunc(display);
 
  glutMouseFunc(mouse);
 
  glutReshapeFunc(reshape);
 
 
 
  glutReshapeWindow(400, 300);
 
 
  printf("%s\n", glGetString(GL_RENDERER));
 
 
  rotation_angle = 30;
 
 
  glutMainLoop();
 
  return 0;
 
}
 
</source>
 
 
== Extension References ==
 
- http://www.opengl.org/registry/specs/ARB/multisample.txt
 
 
These might interest you :
 
- http://www.opengl.org/registry/specs/NV/multisample_filter_hint.txt
 
- http://www.opengl.org/registry/specs/NV/explicit_multisample.txt
 
- http://www.opengl.org/registry/specs/NV/multisample_coverage.txt
 
 
== Conclusion ==
 
Setting up FSAA takes a few steps but it is worth it. Today's GPUs are very rapid and the user wants control over the quality of the game's graphics.<br>
 
On Windows, the extension we are interested in are WGL_ARB_extensions_string which defines wglGetExtensionsStringARB, but the only way to check if this is available is to call wglGetExtensionsStringARB. So this is a chicken and egg situation. Just get the function pointer to wglGetExtensionsStringARB and use it to see if you have WGL_ARB_pixel_format and WGL_ARB_multisample.
 
  
 
[[Category:Sample Writing]]
 
[[Category:Sample Writing]]

Revision as of 14:12, 31 March 2021

Multisampling is a process for reducing aliasing at the edges of rasterized primitives.

Aliasing

Aliasing is a signal processing term caused by analog-to-digital conversions. When an analog signal is converted to a digital signal, this is done by reading the value of the analog signal at a variety of discrete locations called samples. If not enough samples are used, the digital equivalent of the analog signal can manifest unusual patterns. This happens because there are multiple analog signals that could have produced the digital signal using the same sampling pattern. Because the digital signal is not a unique representation of the analog signal, the effects that this produces are called "aliasing" (one signal acts as an "alias" for another).

In terms of computer graphics, 2D rendering is a form of analog-to-digital conversion. The world of primitives is an analog world described mathematically. Rendering it to a series of discrete pixels creates a digital signal representing that world. If not enough samples are used in creating those discrete pixels, the resulting digital image can exhibit aliasing effects.

Aliasing effects in graphics tend to appear as jagged edges of triangles or textures that look extremely rough in areas that should appear smooth, particularly edge on. When animation gets involved, aliasing becomes extremely noticeable, as aliasing pixels tend to shift from color to color abruptly instead of smoothly.

Antialiasing

Combating the visual effects of aliasing is simple, so long as performance is irrelevant. Aliasing is the effect of not using enough samples in analog-to-digital conversions. Thus, all true antialiasing techniques revolve around increasing the number of samples used.

Texture filtering is a form of antialising, applied specifically to aliasing caused by accessing textures. Linear filtering mixes together neighboring samples instead of just using one. Mipmaps of a texture are essentially ways of pre-computing an approximation of accessing a large area of a texture. Each texel in a mipmap represents the average of several texels from the higher mipmap. Anisotropic filtering's ability to access different locations from a texture is also a form of antialiasing, fetching multiple samples to compute a more reasonable value.

But texture filtering only deals with aliasing that results from accessing textures and computations based on such accesses. Aliasing at the edge of primitives is not affected by such filtering.

A more general form of antialiasing is the simplest: render at a higher resolution, then compute the final image by averaging the values from the higher resolution image that correspond to each pixel. This is commonly called "supersampling".

In supersampling, each pixel in the eventual destination image gets its data from multiple pixels in the higher resolution image. The high-res pixels that correspond to a particular destination pixel are called "samples". Given this idea, we can think about a supersampled image as having the same pixel resolution as the destination, but with each pixel storing multiple samples of data.

An image where each pixel stores multiple samples is a "multisampled image".

When we do rasterization with supersampling, the primitive is broken down into multiple samples for each pixel. Each sample is taken at a different location within the pixel's area. So each sample contains all of the rasterization products and everything following them in the pipeline. So for each sample in the multisampled image, we must produce a Fragment, execute a Fragment Shader on it to compute colors, do a bunch of other operations, and write the sample.

As previously stated, each sample within a multisampled pixel comes from a specific location within the area of that pixel. When we attempt to rasterize a primitive for a pixel, we sample the primitive at all of the sample locations within that pixel. If any of those locations fall outside of the primitive's area (because the pixel is at the edge of the primitive), then the samples outside of the area will not generate fragments.

So in every way, supersampling renders to a higher-resolution image; it's just easier to talk about it in terms of adding samples within a pixel.

While this is very simple and easy to implement, it's also obviously expensive. It has all of the downsides of rendering at high resolutions: lots of added rasterization, lots of shader executions, and those multisampled images consume lots of memory and therefore bandwidth. Plus, to compute the final image, we have to take time to average the colors from the multisampled image into its final resolution.

Multisampling

Reasonable use of texture filtering can reduce aliasing within the area of a primitive. As such, supersampling's primary value is in dealing with aliasing at the edges of primitives. But the cost of supersampling affects all parts of rasterization, so the costs tend to outweigh the benefits.

Multisampling is a small modification of the supersampling algorithm that is more focused on edge antialiasing.

In multisampling, everything is set up exactly like supersampling. We still render to multisampled images. The rasterizer still generates (most of) its rasterization data for each sample. Blending, depth testing, and the like still happen per-sample.

The only change with multisampling is that the fragment shader is not executed per-sample. It is executed at a lower frequency than the number of samples per pixel. Exactly how frequently it gets executed depends on the hardware. Some hardware may maintain a 4:1 ratio, such that the FS is executed once for each 4 samples in the multisample rendering.

That brings up an interesting question. If the FS is executed at a lower frequency, how do the other samples get their post-FS values? That is, for the samples that don't correspond to a FS execution, from where do they get their fragment values?

The answer is simple: any FS invocations executed in a multisampled pixel will copy their values to multiple samples. So in our 4:1 example above, if the multisample image contains 4 samples per pixel, then those four samples will get the same fragment values.

In essence, multisampling is supersampling where the sample rate of the fragment shader (and all of its attendant operations) is lower than the number of samples per pixel.

Note however that the depth value is still computed per-sample, not per fragment. This means that, unless the fragment shader replaces the fragment's depth value, the depth value for different samples will have different values. And the sample values in the depth buffers can have different values as well. Depth and stencil tests happen per-sample, and the failure of individual samples prevent the fragment's values from being written to those samples.

Coverage

There still remain a few details to cover. When discussing supersampling, it was mentioned that samples that are outside of the area of the primitive being rasterized don't get values. We can think of the samples in a fragment being rasterized as having a binary state of being covered or not covered. The set of samples in a fragment area covered by the primitive represents the "coverage" of that fragment.

When a depth or stencil test fails for a sample, this modifies the coverage for the fragment by turning off that particular sample for the fragment.

The concept of coverage is important because it can be accessed and manipulated by a fragment shader.

In OpenGL 4.0 or ARB_sample_shading, the fragment shader has a bitmask input that represents the samples covered by that fragment: gl_SampleMaskIn. It also has an output variable that can be used to set the coverage of the fragment: gl_SampleMask. Note that the output sample mask will be ANDed with the physical coverage of the primitive, so you cannot use this to write to samples that are outside of the space actually covered by the primitive.

The alpha value of color number 0 (index 0) generated by the fragment shader can also be set to manipulate the coverage mask (this feature predates gl_SampleMask and is less capable and more hardware-dependent, so you should just modify the sample mask directly if you can). This is activated by using glEnable(GL_SAMPLE_ALPHA_TO_COVERAGE). When activated, the alpha value will manipulate the coverage mask, such that an alpha of 1.0 represents full coverage and a value of 0.0 represents no coverage. The details of the implementation of the mapping of alpha to coverage are hardware-specific, but the mapping is expected to be linear.

The goal of this tool is to make multisampling act somewhat like alpha blending, with values closer to 1 taking up more samples in an area than values closer to 0.

Much like changing the coverage mask directly, alpha-to-coverage will never make the coverage mask affect samples that aren't physically covered by the primitive area.

Because this functionality uses the alpha value being output, that alpha value is not meaningful anymore. If you don't want to use Blending or Write Masking to avoid changes to the alpha, you can glEnable(GL_SAMPLE_ALPHA_TO_ONE). This causes the alpha for the fragment to be converted to 1.0 after the point where it is used to modify the coverage mask.

Primitive edges

There is one major caveat to executing the fragment shader at a rate lower than the number of samples: the location of the FS being executed within the pixel. In multisampling, because the FS invocation will be broadcast to multiple samples, the location of that FS invocation within the pixel will not always match the location of the samples that receive that data.

But more important, it is entirely possible that the sample location used by the FS invocation could be outside of the primitive's area, if the pixel is at the edge of the primitive. This is fine for many FS computations, as interpolation of values past the edge of a primitive can still work mathematically speaking.

What may not be fine is what you *do* with that interpolated value. A particular FS execution may not function correctly in the event that the interpolated values represent a location outside of the primitive area. For example, you might apply a square root to an interpolated value, expecting that the value will never be negative. But interpolation outside of the primitive may force it to be negative.

This is what the centroid interpolation qualifier fixes. Any fragment shader inputs qualified by this identifier will be interpolated within the area of the primitive.

Multisample Resolve

Once a multisample image is built and rendered to, if it is to be viewed, it must be reduced down to a single-sample per-pixel. The process of downsampling a multisample image is called the "multisample resolve". In OpenGL, to perform a multisample resolve, you use a blit operation from a multisampled framebuffer to a single-sampled one. Note that such a resolve blit operation cannot also rescale the image or change its format.

If you create a multisampled Default Framebuffer, the back buffer is considered multisampled, but the front buffer is not. So swapping the buffers is equivalent to doing a multisample resolve operation.

Per-sample shading

Per-sample shading
Core in version 4.6
Core since version 4.0
ARB extension ARB_sample_shading

The whole idea of multisampling is that the FS should execute at a lower frequency than the sample count. However, it is sometimes useful to force the fragment shader to execute at the sample rate. For example, if you are doing post-processing effects on a multisample image before resolving it, then you need to execute those effects on each sample within each pixel. And the results could be very different depending on overlapping primitives and the like.

This is called "per-sample shading", and it effectively transforms multisampling into supersampling.

Per-sample shading is activated for a shader if you use the sample interpolation qualifier on any FS input value, or if you access gl_SamplePosition or gl_SampleID in the FS.

Smooth antialiasing

Before GL_ARB_multisample extension, the edges of lines, polygons, and points could be selectively antialiased using using glEnable(GL_LINE_SMOOTH), glEnable(GL_POLYGON_SMOOTH), glEnable(GL_POINT_SMOOTH) respectively, combined with a blending function, such as glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA). Such features were typically not implemented in hardware in early consumer graphics cards at the time, and were done in software resulting in poor performance. More expensive "workstation" graphics cards from HP, Sun, and SGI at the time did implement these features in hardware.