Credential Engine Registry Handbook
Introduction: Credential Engine Registry
The Credential Engine Registry is a repository of metadata about Credentials and related entities (such as organizations, assessments, and learning opportunities) and a set of related services to enable publishing, finding, and retrieving data about these entities.
This data is described using CTDL, which allows for consistent data across a variety of sources and types of entities.
CTDL is based on linked data principles. Individual CTDL records in the registry are formatted using JSON-LD. The Registry itself is not an RDF triples store, but the records within it can be easily translated into RDF triples.
Data in the Credential Registry is automatically and periodically uploaded to Archive.org for permanent keeping. These records are stored in the same way in which they are received, but are first base64 encoded to eliminate any potential issues with special characters.
Below is some key information that will help you quickly understand the structure and function of data in the Registry.
CTID
As you publish entities, you will receive responses that contain Credential Transparency Identifier (CTID) s. A CTID is a special unique identifier for a top-level object in the Registry, and is the preferred way to reference one entity from another. A CTID is not the same as a version number, nor is it the same as the unique identifier for a particular record in the Registry (the "envelope ID"). The CTID stays with your data as it evolves through updates and new versions, and the only time you should use a new CTID is if you are publishing a new entity, such as a new credential or a new assessment that cannot be substituted for the previous one. You are responsible for determining when it is appropriate to use a new CTID.
A CTID is formed from the combination of a static prefix (ce-) and a standard UUID, and looks like this: ce-6f05953f-8541-4c79-80be-5ce11040f5b6. CTIDs can be readily generated in most programming languages, and you will need to generate these yourself. Ensure that your system keeps track of CTIDs so that it can maintain references to entities in the Registry. You will not be able to update or remove records in the Registry without the CTID!
In order to reference an entity in the registry, you will need to combine the Registry's resources URL (https://sandbox.credentialengineregistry.org/resources/) with the CTID. Together, these form the unique resource identifier (URI) for your entity, which looks like this: https://sandbox.credentialengineregistry.org/resources/ce-6f05953f-8541-4c79-80be-5ce11040f5b6. Resolving the URI directly will yield the latest version of the data for that CTID (as stated, it is not tied to a particular version). Such URIs are used within the Registry to enable entities to point at each other, so that no entity needs to contain or maintain the actual data for another entity (for example, a Credential never contains the data about the learning opportunities it requires; it only contains references to them via their URIs, which return that data when resolved).
Organization Accounts and Published Organization Data
Credential Engine relies on other organizations as primary sources of the information in the Credential Registry. Part of enabling those organizations to publish data to the Registry is issuing and maintaining accounts for those Organizations. While there is a great deal of overlap in the data for an Organization Account in Credential Engine and the data about an Organization that gets published to the Registry using CTDL, these two objects are separate things with distinct purposes and it is important not to confuse one for the other. In other words, an Organization must first create an Organization Account for itself with Credential Engine and then publish an Organization record about itself, using CTDL, to the Registry. However, these are two entirely separate representations of the same Organization.
The Credential Registry and CTDL
The Credential Registry implements some additional constraints that are not part of CTDL. These additional constraints make up the Registry's "Application Profile" of CTDL. The following are a few examples of constraints on data in the Registry that are not part of CTDL:
- All credentials must have a name and description.
- The Person class is never used, since the Credential Registry does not contain any personal information or records dealing with individuals.
- The keyword property allows multiple values, while subjectWebpage, in most cases, does not.
- The description for a credential must have at least 25 characters.
- The only valid value for the requires property is an array of Condition Profile objects.
- Credential Engine checks for valid subjectWebpage URLs, but does not force a specific format for the telephone property, since different parts of the world have different formats for their phone numbers.
- The Credential Alignment Object is always embedded as a nested object in the JSON data wherever it appears.
- The value of a credential's accreditedBy property, if it is present, is always an array of strings where each string is either a URI that resolves to a separate record or a blank node ID that resolves (within the graph for that credential) to a blank node that describes the accrediting organization.
- Records in the registry are considered to be from an authoritative source, except for blank nodes, which are non-authoritative and enable useful assertions like the one above about accreditation. Wherever possible, an authoritative record should be referenced by its URI instead of creating a blank node.
For a more detailed listing of the constraints in place for the Registry, consult the Policy Page.
Object Types
The Credential Registry supports several "top level" or "main" CTDL classes that are considered to be standalone and can be individually published to and retrieved from the Registry. All other CTDL classes must be embedded inside of one of these classes when being published to the Registry. These top level classes are:
Types of Organization
Types of Credential
Assessments
Learning Opportunities
Manifests
Competency Frameworks
Concept Schemes
Planned
Get Started
In order to publish or consume data, you must first register your organization:
- Navigate to the Credential Engine Accounts site.
- Create an account. After registering, you will receive an email to confirm your account.
- After confirming your account, add your organization.
- Complete the required information for the organization, including the publishing and/or consuming methods you want to use.
- Submit your organization. Credential Engine will review and approve your organization.
Get Your API Key
Most of the ways you interact with the Registry will require an API key. The Get Record and Finder Widget search methods are notable exceptions. You can find your API key on your Credential Engine Account after your organization has been approved to publish or consume from the Registry. Note:
- The same API key can be used for all functions that require an API key.
- You should not reveal your API key to any other organization or user, or in any public place.
- You can generate a new API key at any time from your dashboard.
- If you wish to have another organization publish on your behalf, or if you wish to publish for another organization, use the functionality built into the account system to handle this. Do not share API keys.
Publish Data
Publishing data requires several things:
- You must sign up for a Credential Engine Account
- Your account must be authorized to publish data
- You must obtain an API key
- You must be able to provide data that conforms to the Minimum Data Policy
- You must keep track of the data you publish in case you want to update or delete it later
Manual Entry
For organizations with a small number of credentials that aren't updated very often, the manual publishing tool is likely the right choice. Authenticated users with approved organizations can access the manual publishing tool via the Credential Publisher website.
Using the tool is straightforward: Just click the "Add New" button in the Credential Publisher's header and select the type of data you want to publish. Then work your way through the forms presented, filling in as much information as you have available to you.
At the end, click the "Save Data" button, then click "Review" to see a summary of the information you entered. If you're happy with it, click the "Approve" button in the review interface. The Credential Engine staff will be notified, and conduct a final review of your data. If it meets the requirements and there aren't any issues, Credential Engine will then publish the data to the registry.
The Manual Entry tool will also enable you to update your data later.
Spreadsheet-Based Bulk Upload
Organizations with a large number of credentials but limited access to information technology development staff may find the bulk upload option to be the best bet. This tool can be accessed via the Bulk Upload Page on the Credential Publisher website.
To use this tool, you will need to be able to provide your credential information in a spreadsheet in CSV format, with columns that match up to what is provided to you by the instructions on the bulk upload page. You can download a spreadsheet template and fill it in manually, or export the data from your system. You can also update via the bulk upload tool.
Refer to the bulk upload page itself for detailed instructions and guidance.
Registry Assistant API
Organizations with a large number of credentials (and/or credentials that are updated frequently) and which have ready access to information technology development staff may find the Registry Assistant API to be the way to go. The Registry Assistant API Handbook should be the starting point for using this tool.
To use the Registry Assistant API, your system will need to be able to provide the data according to the schema described on the API Handbook page, and you will also need the API key from your dashboard in the Credential Engine Accounts System. It also helps if your development team is familiar with JSON and CTDL. It is also possible to update your data via the Assistant API.
For complete details on using the publishing API see section Registry Assistant API Handbook
Consume Data
In most cases, you will be coming to the Registry to find data. There are multiple ways to do so, depending on your needs. Each of these methods requires an approved Credential Engine Organization Account.
- Get the record directly via its CTID if you already know the CTID for the data you want.
- Use a customized Search Widget provided by the Credential Finder.
- Use the CTDL JSON-based Search API to build a system that can perform complex searches.
- Download data in bulk to store and use offline within your system.
Get Record
You can get a record from the registry directly by CTID. There are two options:
-
Using /graph - to return the full document and any child entities such as blank nodes, or competencies for a competency framework
https://sandbox.credentialengineregistry.org/graph/{CTID} -
Using /resource - to return only the main document and no child documents.
https://sandbox.credentialengineregistry.org/resources/{CTID}
For example:
https://sandbox.credentialengineregistry.org/graph/ce-8a82436e-49bf-4856-b67a-60f1984df77d1
Finder Widget
The Credential Finder hosts customizable Search Widgets that can be embedded in your website to provide your visitors with a way to search the data with very minimal technical know-how or investment.
Some examples of how a widget can be customized to filter search results are:
- By Location (Country, Region, or City)
- By Owning organization(s) - only show artifacts owned by the selected organization(s)
- By relationships such as Offered By, Accredited By, Approved By, Regulated By - only show artifacts with the related relationship with the selected organization(s)
Sample Widgets
Indiana
The Indiana widget uses filters to only show credentials where the owning organization has an address in Indiana.
NOCTI
The NOCTI widget uses filters to only show credentials where the owning organization is NOCTI.
Bulk Download for Offline Use
The Credential Engine Registry offers a way to quickly download all of the data in the registry for use by your system directly. This allows for applications that may need different access to the Credential Registry data than what the Search API provides.
Note:
- All of the options below require that your organization be approved to consume data for "Offline Storage" in the Credential Engine accounts system.
- You will need to periodically update the data, as the data in the Registry changes daily.
JSON-LD in a single zip file
This feature allows you to download the resources from the Registry as individual JSON-LD files through a single .zip file.
To access it, log into your Credential Engine account, click the "Download Data" tab from your organization's dashboard page, and click the "Download All Credential Registry Data" button.
JSON-LD in separate files
This feature enables you to run a lightweight automated process for downloading updated JSON-LD files from the Registry on a regular schedule. It can be configured to only download data that has changed, making it easier to keep your system up to date.
For documentation about using this feature, see the Github Repository.
Transformed data in a SQL Database
This feature offers the same functionality as above, but will convert the downloaded data to a ready-made SQL data structure for direct use by your system.
For documentation about using this feature, see the Github Wiki for the Credential Registry Import Tool.
Search API
The Credential Engine Registry offers a Search API to enable the development of Credential Registry search applications.
In essence, the Search API works like this:
- Your system constructs a JSON document that mirrors the structure of CTDL records in the Registry.
- This document contains only the properties you wish to query on; all other properties are omitted.
- This document may contain special operators to modify the behavior of the search for specific properties (such as requiring an exact match for text fields).
- The order of properties and values does not matter, but the structure of the objects in your query does.
- You will need to be aware of properties whose values use language maps instead of simple strings.
- In general, properties contained within the same object are searched for with logical ANDs between them, while items in arrays are searched for with logical ORs between them. This behavior can be overridden if desired.
- Your system passes the JSON document and your API key along with properties to enable paginating through result sets to the Search API endpoint.
- The results will be passed back to your system as described below.
Preparing to use the Search API
Before you begin using the search API, you will need:
- An approved Credential Engine organization account - Visit the Credential Engine Accounts System to get signed up.
- An API Key for your organization - You can find this on your account dashboard after your organization is approved to search.
- An understanding of CTDL - Review the CTDL Handbook and CTDL Explorer.
Calling the Search API
To call the search API, you will need to make an HTTP POST request to the API endpoint:
https://apps.credentialengine.org/assistant/search/ctdl
With your request, include two things: First, you will need to pass your organization's API key as a request header using the format:
Authorization: ApiToken [YOUR API KEY]
Second, the body of the request must be a JSON object with the following properties:
- Query
- The CTDL JSON query itself.
- Skip
- An integer representing how many results to skip over before the value provided by Take is applied. Enables paging through large result sets. Defaults to 0.
- Take
- An integer representing how many results from the current result set to take after the value provided by Skip. Enables paging through large result sets. Defaults to 5. Maximum 100.
An example HTTP POST body would look like this:
{
"Query": { "ceterms:name": "My credential's name" },
"Skip": 0,
"Take": 10
}
It is recommended to use the Query Helper to develop, test, and debug queries before implementing them in your system's code.
CTDL JSON Query Structure
Since relevant, searchable data can exist in many different objects and places in CTDL, it is important to look for the correct data in the correct place. For instance, you would not look for the required assessments for a credential within the body of the credential itself; you would need to follow the CTDL structure as it appears in the Registry. Below are some examples that show this structure in use.
It is important to understand that multiple object types coexist in the Registry, and you will likely need to filter by these types to ensure you don't get assessments or learning opportunities when you're looking for credentials.
Additionally, unless noted otherwise, your queries will need to match the structure of the CTDL classes and use the correct data types for properties (strings, numbers, booleans, etc).
For more examples and to try out your own customized queries, use the CTDL Query Helper tool.
Special Search Terms
The search API supports several special search terms that can be combined with the CTDL schema to fine-tune how queries function:
The following properties are supported:
search:value- Sometimes you will want to change how a plain value (such as a string or number) is queried. Any time a property would normally take a value (such as a string, number, or boolean), you can substitute that value for an object. Inside that object, use
search:valueto convey the value, and addsearch:operatorto the object to adjust how that value is queried. - Accepts: A single value, an array of values, a single object, an array of objects
search:operator- In cases where
search:valueis present in the same object as this property, use this property to control how the values insearch:valueare queried. Otherwise, this will control how values in the object this property appears in are queried. See the list of supported values below for more information. - Accepts:
search:andTerms,search:orTerms search:termGroup- This term enables you to look for specific combinations of properties and values, even if they would otherwise overlap with each other. When combined with
"search:operator": "search:orTerms", it also enables searches that need to find a piece of information in at least one property. See below for examples of usage. - Accepts: A single object, an array of objects
The following values are supported:
search:andTerms- You can use
search:andTermsas the value forsearch:operatorto force the items in thesearch:valuearray to be queried using a logical AND instead of a logical OR (logical OR is the default handling for values in an array). - Value For:
search:operator search:orTerms- You can use
search:orTermsas the value forsearch:operatorto force the items in any object to be queried using a logical OR instead of a logical AND. (logical AND is the default handling for values in an object). - Value For:
search:operator search:anyValue- If you don't care what the value for a property is, and simply want to see if a piece of data has that property at all, use
search:anyValueas the value for the property you're looking for. - Value For: Any CTDL/CTDL-ASN property
Simple Queries
The following examples demonstrate basic search functionality by examining fields that exist at the top level of the object.
ceterms:Certificate:Deeper Queries
These queries show examples of traversing other objects within the top-level object.
ceterms:requires property)audience:FormerMilitary. Note that because the query mirrors the structure of CTDL, this query includes the structure of the ceterms:CredentialAlignmentObject used for Concept Schemes.ceterms:ConditionProfile) are in an array together. Items in arrays are, by default, queried as logical ORs. Objects are, by default, queried as logical ANDs, so this query could be thought of like this:Find credentials where ( @type == ceterms:Certificate OR @type == ceterms:Certification ) AND ( ( requires the provided description AND requires the two conditions ) OR ( requires 3 years of experience ) )
Overriding Default AND/OR Operators
Sometimes it's useful to override the default behavior of objects (AND) and arrays (OR). The search API uses a pair of special properties for this: search:value to provide the values, and search:operator to control whether those values are compared via AND (search:andTerms) or OR (search:andTerms). The following examples illustrate the usage of these properties.
ceterms:subjectWebpage value that matches any of the provided values. This takes advantage of the fact that the default handling for arrays, which is to use logical ORs.If you need to provide multiple values for a single property, use an object as the value for that property and, inside that object, use search:value to contain the desired values. You can then use search:operator to control how those values are evaluated (the default is to use logical ORs since the contents of search:value are an array).
ceterms:inLanguage property contains all of the provided languages, instead of simply looking for a match on any of them.This is done by using
search:value to contain the desired values and search:operator to dictate how to compare those values.The next set of examples illustrate one of the nuances of using the CTDL structure to query for data. The first example will match a credential offered by any organization whose CTID is either of the provided values - a logical OR. If you were to use "search:operator": "search:andTerms" with this query, you would be looking for a credential offered by an organization whose CTID matches both values - and would get no results!
Instead, to look for a credential that is offered by both organizations, you need to look for two separate organizations as values for the ceterms:offeredBy property. Using "search:operator": "search:andTerms" here will look for a credential whose ceterms:offeredBy property contains both values.
Sometimes it's necessary to find credentials where some piece of data appears in one or more properties, without requiring that it appears in all of them, or to look in certain combinations of properties. For these cases, use search:termGroup:
search:termGroup. You can use it to look for matches in different combinations of properties, with different operators for each combination.search:termGroup can also look for different combinations of values for the same properties.Language Maps
All langauge-dependent string fields in the Registry use Language Maps to contain data. Language maps looks like one of the below examples, depending on whether or not the term supports multiple values. Note that for multi-value language maps (as shown below via ceterms:keyword), the number, order, and meanings of values are not necessarily identical across languages.
"ceterms:name": {
"en": "The name of the Credential in non-region-specific English",
"en-us": "The name of the Credential in the US",
"en-gb": "The name of the Credential in Britain",
"fr": "The name of the credential in French",
"ru": "The name of the credential in Russian"
},
"ceterms:keyword": {
"en": [ "English keyword 1", "English Keyword 2", "English Keyword 3" ],
"fr": [ "Keyword in French that covers the meanings of the first two english keywords", "Keyword exclusive to the list of French keywords" ],
"ru": [ "Russian keyword 1", "Russian keyword 2", "Russian keyword 3", "Russian keyword 4"]
}
Each property in a language map represents a BCP 47 language code that may or may not have a region code attached. A language map may have any number of these properties (though it will have at least one). This means that data in the registry may or may not have text in the language(s) that your system looks for.
The search API enables treating language maps like any other object in terms of which properties you query against. However, in order to avoid needing to send large queries that explicitly list the same text string for every potential combination of language and region code your system might accept, the search API supports special handling for properties that use language maps. This handling is detailed below.
The CTDL properties that use language maps are:
The following examples illustrate various ways to query against language map properties:
ceterms:name field without specifying what language(s) to check. If the provided text matches the name in any language, it will be considered a match.Note that if a credential had the provided text in an
en-us field instead, it would be missed by this query.Note that because
ceterms:name is provided as an object here, the languag ecodes are AND-ed together. This means that every provided language must match in order for the credential to be returned by the search.Note also that this is redundant if your query is looking for the same text in each of the provided languages - this redundancy can be avoided by following the "easy" example below.
Note that this is redundant if your query is looking for the same text in each of the provided languages - this redundancy can be avoided by following the "easy" example below.
Note that because
ceterms:keyword is provided as an object here, the language codes are AND-ed together. This means that every provided language must have at least one matching keyword in that language in order for the credential to be returned by the search.Controlled Vocabularies
CTDL uses a variety of controlled vocabularies (RDF Concept Schemes) to convey a wide variety of information in a way that is easy and reliable to reference. These references use ceterms:CredentialAlignmentObjects to reference concepts in a way that allows some data about the concept to be stored within the credential's record, for ease of use and consumption (and to handle various edge cases that are beyond the scope of this guide). It's important to be aware of the presence of ceterms:CredentialAlignmentObject when constructing CTDL JSON queries.
The next query illustrates why it's important to remember the nature of ceterms:CredentialAlignmentObject and its usage when pointing to terms in a vocabulary. ceterms:CredentialAlignmentObject only has one value for its ceterms:targetNode property.
ceterms:CredentialAlignmentObject, and will therefore fail. This is an example of what not to do.In order to require a match for multiple vocabulary terms, you will need to construct your query such that each term has its own ceterms:CredentialAlignmentObject as illustrated below.
ceterms:audienceType property, via multiple instances of ceterms:CredentialAlignmentObject.Searching Across Objects
The Credential Registry contains a vast collection of "top-level" objects. These are objects that have their own URIs and can be referenced directly in the Registry. The CTDL/CTDL-ASN top-level objects in the Registry are:
It is often useful to query for one type of top-level object based on its connections to another top-level object. For instance, retrieving a list of all credentials offered by an organization that operates in a certain state, or finding all credentials that require a certain assessment, or all learning opportunities that teach a competency that contains a certain phrase. These objects all exist on their own in the registry, but the search API is configured to allow you to query for these objects as though they exist within each other. See below for some examples.
ceterms:AssessmentProfiles. You can use any kind of query for data in the top-level objects together - just make sure you look for the right things in the right places.ceasn:competencyText property contains the provided text.Note the need to account for the presence of the
ceterms:CredentialAlignmentObject as the value for ceterms:targetCompetency.Note the use of both CTDL (via
ceterms:) and CTDL-ASN (via ceasn:) here.References
- Credential Engine Registry Github Repository: https://github.com/CredentialEngine/CredentialRegistry
