Mouse Selection using shader buffers

For this technique to work you need at least OpenGL 4.3, but that said even quite long in the tooth integrated graphics can do this (at least on Linux, and I guess on windows), SSBO's (Shader Storage Buffer Objects) are a great way for your shader to feedback information to your application.

In this case when the shader reaches a specific screen position, I'm saving the nearest ID of which ever model is currently being rendered.

Looking at the fragment shader first we define a few things

// redefine px coord
layout(origin_upper_left, pixel_center_integer) in vec4 gl_FragCoord;

layout (std430, binding=0) buffer id_data
{
    uint id;
    float depth;
};

The buffer contains just two items, the id of the model that's found and we keep track of the depth too, this is because we only want information about the closest model that's rendered, look at the painters algorithm to see why this is important.

Later on in the body of the fragment shader:

    if (gl_FragCoord.x==960 && gl_FragCoord.y==540) {
        if (gl_FragCoord.z < depth) {
            id = mod_id;
            depth = gl_FragCoord.z;
        }
    }

Now as I haven't quite finished my implementation just yet, these magic values are simply the centre coordinates of the fixed render texture I render everything to, but you can see how these could easily be a uniform updated with the current mouse position each frame... (I'm mainly using mouse look, but I will be needing other methods later)

Once the rendering is happening at the point we are interested in, its just a case of checking the saved depth value to see if the current fragment is nearer, if it is save the model ID and update the depth value.

That's it for the shader, but the application has some work to do too. As I'm using Raylib and also mixing it with "raw" OpenGL functions, its necessary to include the GL prototypes

#define GL_GLEXT_PROTOTYPES 1
#include <GL/glcorearb.h>

Next we need somewhere to store the applications copy of the buffer the shader will be using.

typedef struct _info {
    unsigned int id;
    float depth;
} _info;

_info info;
GLuint ssbo = 0;

The ssbo variable is used to identify the SSBO itself

    int mod_id = GetShaderLocation( shader, "mod_id" );
    
    // set up the data buffer
    glGenBuffers(1, &ssbo);
    glBindBuffer(GL_SHADER_STORAGE_BUFFER, ssbo);
    glBufferData(GL_SHADER_STORAGE_BUFFER, sizeof(unsigned int)+sizeof(float), &info, GL_DYNAMIC_COPY);
    glBindBuffer(GL_SHADER_STORAGE_BUFFER, 0);

    GLuint block_index = 0;
    block_index = glGetProgramResourceIndex(shader.id, GL_SHADER_STORAGE_BLOCK, "id_data");
    TraceLog(LOG_INFO,"SSBO index %i",block_index);
    GLuint ssbo_binding_point_index = 0;
    glShaderStorageBlockBinding(shader.id, block_index, ssbo_binding_point_index);
    glBindBufferBase(GL_SHADER_STORAGE_BUFFER, ssbo_binding_point_index, ssbo);   

Now we have the SSBO set up and a uniform we can set with a model ID as we render each model. Each frame before we do any rendering we need to reset the values in the buffer

        // clear the id
        glBindBuffer(GL_SHADER_STORAGE_BUFFER, ssbo); 
        void* p = glMapBuffer(GL_SHADER_STORAGE_BUFFER, GL_WRITE_ONLY);    
        ((_info*)p)->id = 0;
        ((_info*)p)->depth = 1;
        glUnmapBuffer(GL_SHADER_STORAGE_BUFFER);

We map the buffer and set the id to zero (meaning no model) and set the depth value to maximum (range 0-1). Now we are all set up ready to render

        glUseProgram(shader.id);
        glUniform1ui( mod_id, 1);
        DrawModel( model, (Vector3){0,0,0}, 1, WHITE );

        glUseProgram(shader.id);
        glUniform1ui( mod_id, 2);
        DrawModel( model, (Vector3){10,0,0}, 1, WHITE );

Its probably more useful to set the model id to an array index which will give you a pointer to a structure of information for each object you render, so for example you might be tracking (and editing) size, orientation and other properties for each model within your application. As I'm using an unsigned integer I've resorted to directly using GL to set the uniform as there isn't support in Raylib for unsigned uniforms.

Once rendering is complete the ID of any model at the selected spot can be retrieved

        // copy data from the shader     
        glBindBuffer(GL_SHADER_STORAGE_BUFFER, ssbo); 
        p = glMapBuffer(GL_SHADER_STORAGE_BUFFER, GL_READ_ONLY);
        info.id = ((_info*)p)->id;
        glUnmapBuffer(GL_SHADER_STORAGE_BUFFER);

That's just about it, using an SSBO is a very useful technique and it certainly beats encoding integer ID's into RGB floats and back!

Enjoy!