Construction libraries are created from the opening screen by clicking on the "Component libraries" tab. Then click on the + to the left of Constructions to open the list of construction categories and click on the category of construction you wish to define.  Then click on the + toolbar icon to create the construction.  If you have not already defined the thermophysical properties of each layer you can do this by clicking on the + in the material selection dialog.

All components in the library including the ones you create are copied to new files at the point they are created. Any changes you make to model data components are not reflected in the equivalent library data (unless you explicitly copy your model data components to the library).