Platform FAQ
From n² wiki
[edit] Introduction
[edit] What is the Talis Platform?
The Talis Platform is an environment for building next generation applications and services based on Semantic Web technologies. It is a hosted system which provides an efficient, robust storage infrastructure. Both arbitrary documents and RDF-based semantic content are supported, with sophisticated query, indexing and search features.
see also: http://www.talis.com/newsletters/platform/ and http://www.talis.com/tdn/node/1727 (less up-to-date than this wiki)
[edit] What is bigfoot?
Bigfoot is the component at the core of the platform, it takes care of managing the stores.
[edit] Is the Platform going to be open source?
We'll certainly open source lots of the platform and obviously we're using open source software extensively. No final decision has been made on open sourcing the entire platform codebase. It's mainly about timing: the platform is still evolving so we want to be able to change things radically if we go down a blind alley. Making it OSS too soon might mean we have to fork it just to repair something we did wrong.
[edit] Who will own the data I put into the system?
People using the platform will always own their own data. They will always be able to remove their data from the platform - it's a core platform principle. We'll also provide ways for them to declare their own licencing terms etc. We won't put obstacles in their way if they want to leave, so that means we have to be better than the alternatives so they don't want to leave.
[edit] What's the business model for the Platform?
todo see also http://lpag-wiki.talis.com/wiki/Platform_FAQ#How_much_does_a_store_cost.3F
[edit] How many triples?
todo
[edit] Where can I find out more about the Talis Platform?
You can visit the Talis Platform Website which contains more information. We recently published a White Paper which explains some of the reasoning behind creating the platform. We also blog regularly on Nodalities, a group blog for all things Platform related.
[edit] Platform Stores
[edit] About Stores
[edit] What is a store?
A store is a grouping of related data and metadata. For convenience each store is assigned one or more owners who are the people who have rights to configure the access controls over that data and metadata. Each store provides a uniform REST interface to the data and metadata it manages.
A store consists of two "boxes" – a Contentbox designed for unstructured or binary blobs and a Metabox designed for holding richly structured metadata. The metadata in the metabox may be about the content in the contentbox or it may be unrelated. The metabox is implemented as an RDF triple store. As content is added to the contentbox metadata is automatically extracted from it and stored in the associated metabox. The platform allows querying of the metabox directly using Sparql or of the contentbox using store-specific indexes. Stores can be aggregated in a number of configurations to allow querying across groups of stores.
[edit] What is a Contentbox?
The Contentbox is the part of a store designed for holding unstructured data. You can access a store's contentbox at: http://api.talis.com/bf/stores/{storename}/items
You can POST things to it and rules can be configured per store to extract metadata from those items. You can also search it using GET. The search results are returned in RSS 1.0 by default.
[edit] What is a Metabox?
The Metabox is the part of a store designed for holding structured metadata in the form of an RDF graph. The URI of each store's Metabox is http://api.talis.com/stores/{storename}/meta
The Metabox can be searched using Sparql. RDF can be added to it by POSTing either an RDF/XML document or a Changeset
[edit] Creating a Store
[edit] How do I create a store?
Currently we are allocating stores on a restricted request basis. In the future we will allow self-service creation of stores.
[edit] How much does a store cost?
If you are a member of the n2 developer community then you may have any number of free development stores. Please contact danny.ayers@talis.com to request a store.
We can offer free unlimited use stores for non-profits and open source projects. Please contact danny.ayers@talis.com or paul.miller@talis.com if you think you're eligible.
We also provide stores for commercial use. These are provided on an experimental basis subject to capacity. We do not currently charge for store creation or ongoing usage costs. However we may choose to in the future. Please contact paul.miller@talis.com for more information.
[edit] Can a store have any name I choose?
Store names may contain only the letters a-z, numbers 0-9 and hyphens. They may only start with a letter. Store names are always lowercase. We allow any name that follow those rules and are not offensive or potential trademark violations.
[edit] Using a Store
The platform is a collection of stores, each store is a set of Ground RDF Graphs (Glossary#Ground RDF Graph) and a Content Store for storing arbitrary documents.
[edit] How do I add RDF to a store?
You can POST RDF/XML directly to a store's Metabox. You must have the full-update capability to perform this operation. Don't forget to set the content-type of the POST to be application/rdf+xml. The following cURL command demonstrates how to add RDF to a store:
curl -d @data.rdf -H content-type:application/rdf+xml http://api.talis.com/stores/mystore/meta
If the store owner has assigned users the full-update capability then you will need to supply a username and password:
curl -d @data.rdf --digest -u "user:password" -H content-type:application/rdf+xml http://api.talis.com/stores/mystore/meta
Note that all URIs in any RDF/XML that you wish to POST must be absolute. Best practice is to specify an xml:base attribute in the RDF/XML to ensure that any relative URIs can be resolved to absolute ones.
Be careful to not include a trailing / after the /meta. This will result in a HTTP response of 401 Unauthorized, but without verbose mode (-v) turned on in curl you won't see an immediate difference.
[edit] How do I update the RDF in a store?
Each store supports the Changeset Protocol for adding and removing triples from the store's Metabox. Changeset can be applied in an unversioned fashion (which requires the full-update capability) or in versioned fashion (which requires the versioned-update capability). Versioned changesets are applied in the same way as unversioned ones but the changeset is retained and linked to previous changesets for the same resource, forming a linked list of changes to a store's Metabox. This list of changesets is like a history of changes to the graph and allows the store owner to create applications that support wiki-like rollback and recent changes lists.
The following is an example of a changeset:
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:cs="http://purl.org/vocab/changeset/schema#">
<cs:ChangeSet>
<cs:subjectOfChange rdf:resource="http://example.com/res#thing"/>
<cs:createdDate>2006-01-01T00:00:00Z</cs:createdDate>
<cs:creatorName>Anne Onymous</cs:creatorName>
<cs:changeReason>Change of title</cs:changeReason>
<cs:removal>
<rdf:Statement>
<rdf:subject rdf:resource="http://example.com/res#thing"/>
<rdf:predicate rdf:resource="http://purl.org/dc/elements/1.1/title"/>
<rdf:object>Original Title</rdf:object>
</rdf:Statement>
</cs:removal>
<cs:addition>
<rdf:Statement>
<rdf:subject rdf:resource="http://example.com/res#thing"/>
<rdf:predicate rdf:resource="http://purl.org/dc/elements/1.1/title"/>
<rdf:object>New Title</rdf:object>
</rdf:Statement>
</cs:addition>
</cs:ChangeSet>
</rdf:RDF>
This changeset corresponds to the removal of a single triple:
<http://example.com/res#thing> <http://purl.org/dc/elements/1.1/> "Original Title" .
and the addition of the following triple:
<http://example.com/res#thing> <http://purl.org/dc/elements/1.1/> "New Title" .
When posting the changeset you must use a content type of application/vnd.talis.changeset+xml
For example to POST an unversioned changeset:
curl -d @changeset.rdf -H content-type:application/vnd.talis.changeset+xml http://api.talis.com/stores/mystore/meta
Or a versioned one:
curl -d @changeset.rdf -H content-type:application/vnd.talis.changeset+xml http://api.talis.com/stores/mystore/meta/changesets
See also ChangeSet TroubleShooting
[edit] How do I add content to a store?
You can POST documents and binaries to a store's Contentbox. You must have the full-update capability to perform this operation. The store will remember the content type of the content so don't forget to specify it in your POST request. The following cURL command demonstrates how to add content to a store:
curl -d @myfile.jpg -H content-type:image/jpeg http://api.talis.com/stores/mystore/items
And the secure version
curl -d @myfile.jpg --digest -u "user:password" -H content-type:image/jpeg http://api.talis.com/stores/mystore/items
[edit] How do I get a list of all the items in a store?
To get a list of all the items submitted to a store, use the Store OAI Service. A URL like this should work to get you started:
http://api.talis.com/stores/mystore/services/oai-pmh?metadataPrefix=oai_dc&verb=ListRecords
[edit] How do I remove items from a store?
If you have the full-update capability then you can issue an HTTP DELETE request to the item's URI. For example:
curl -X DELETE --digest -u "user:password" http://api.talis.com/stores/mystore/items/909332aa-1139-4f00-886f-4b50067e16d4
[edit] Where did all my blank nodes go?
Due to current limitations in the update-protocol (using changesets) the platform does not yet support blank nodes. The Platform replaces graph containing blank nodes with an arbitrary ground instance by generating URIs for all blank nodes added to a store. An implication of this, is that when the same graph is posted multiple times new nodes (additional triples) are created. Another implication is that your RDF is not round-trippable.
[edit] How do I authenticate?
[edit] Why am I getting a SAXParseException when posting to my store even though XML is well-formed?
You may see the following error: rethrew: org.xml.sax.SAXParseException: Premature end of file.
This often happens when using cURL from the command line and you supply user credentials when they're not needed. Try the command again without the authentication type, username or password
[edit] Managing a Store
[edit] What is a Query Profile?
A Query Profile is a list of field names and their weights. It is used to configure the relevance ranking of searches against the Contentbox. When a user searches the Contentbox and doesn't supply an explicit field name in their search the store uses the Query Profile to determine which fields should be searched.
In this example query profile the field "name" is given a weight of 10 and the field "nick" is given a weight of 5:
<?xml version="1.0" encoding="utf-8"?>
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#" xmlns:bf="http://schemas.talis.com/2006/bigfoot/configuration#" xmlns:frm="http://schemas.talis.com/2006/frame/schema#">
<bf:QueryProfile rdf:about="http://api.talis.local/stores/mystore/indexes/default/queryprofiles/default">
<rdfs:label>default query profile</rdfs:label>
<bf:fieldWeight>
<rdf:Description rdf:about="http://api.talis.com/stores/mystore/indexes/default/queryprofiles/default#name">
<bf:weight>10.0</bf:weight>
<frm:name>name</frm:name>
</rdf:Description>
</bf:fieldWeight>
<bf:fieldWeight>
<rdf:Description rdf:about="http://api.talis.com/stores/mystore/indexes/default/queryprofiles/default#nick">
<bf:weight>5.0</bf:weight>
<frm:name>nick</frm:name>
</rdf:Description>
</bf:fieldWeight>
</bf:QueryProfile>
</rdf:RDF>
When a user searches for "bob" the store will rewrite that query internally to be "name:bob nick:bob" and will value occurences of bob in the name field twice as high as occurences in the nick field.
[edit] How do I update my store's query profile?
Use a PUT on the query profile's URI to replace it. You must have the configure-store capability to perform this operation.
For example:
curl -v -T queryprofile.xml -H content-type:application/rdf+xml http://api.talis.com/stores/mystore/indexes/default/queryprofiles/default
[edit] Where is my store's query profile?
A store may have multiple indexes, each of which may have multiple query profiles. For each index one of of the query profiles is nominated to be the default one. The location of this depends on the store's configuration. The definitive way to find out is to query the config for the store, but this is not yet available for use by anyone other than the platform administrator (we hope to have it available for store owners very soon).
Some common locations to try:
http://api.talis.com/stores/mystore/indexes/default/queryprofiles/default http://api.talis.com/stores/mystore/indexes/metaboxIndex/queryprofiles/default http://api.talis.com/stores/mystore/indexes/m21Advanced/queryprofiles/default http://api.talis.com/stores/mystore/config/queryprofiles/1
If none of those work, please contact us and we'll look it up for you.
[edit] How do I view my store's query profile as RDF?
Append ?output=rdf to the URI or supply application/rdf+xml in the HTTP accept header that you send with your request.
For example:
curl -v -H accept:application/rdf+xml http://api.talis.com/stores/mystore/indexes/default/queryprofiles/default
[edit] What is a Field/Predicate Map?
A Field Predicate Map is a list of RDF property URIs and short names that they map to. When a user performs a query on a store's Contentbox the Field/Predicate map is used to convert the short names used in the query into the property URIs that are being searched.
In this example the property URI http://xmlns.com/foaf/0.1/name is mapped to "name" and the URI http://xmlns.com/foaf/0.1/nick is mapped to "nick":
<?xml version="1.0" encoding="utf-8"?>
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#" xmlns:bf="http://schemas.talis.com/2006/bigfoot/configuration#" xmlns:frm="http://schemas.talis.com/2006/frame/schema#" >
<bf:FieldPredicateMap rdf:about="http://api.talis.com/stores/mystore/indexes/default/fpmaps/default">
<frm:mappedDatatypeProperty>
<rdf:Description rdf:about="http://api.talis.com/stores/mystore/indexes/default/fpmaps/default#name">
<frm:property rdf:resource="http://xmlns.com/foaf/0.1/name"/>
<frm:name>name</frm:name>
</rdf:Description>
</frm:mappedDatatypeProperty>
<frm:mappedDatatypeProperty>
<rdf:Description rdf:about="http://api.talis.com/stores/mystore/indexes/default/fpmaps/default#nick">
<frm:property rdf:resource="http://xmlns.com/foaf/0.1/nick"/>
<frm:name>nick</frm:name>
</rdf:Description>
</frm:mappedDatatypeProperty>
</bf:FieldPredicateMap>
</rdf:RDF>
Note that at present, only properties listed in a store's Field/Predicate map are indexed for searching via the Contentbox and that only literal values are indexed.
[edit] How do I update my store's field/predicate map?
Use a PUT on the field/predicate map's URI to replace it. You must have the configure-store capability to perform this operation.
For example:
curl -v -T fpmap.rdf -H content-type:application/rdf+xml http://api.talis.com/stores/mystore/indexes/default/fpmaps/default
Note that updating the Field/Predicate map does not cause existing data to be reindexed.
[edit] Where is my store's field/predicate map?
A store may have multiple indexes, each of which may have multiple field/predicate maps. For each index one of of the field/predicate maps is nominated to be the default one. The location of this depends on the store's configuration. The definitive way to find out is to query the config for the store, but this is not yet available for use by anyone other than the platform administrator (we hope to have it available for store owners very soon).
Some common locations to try:
http://api.talis.com/stores/mystore/indexes/default/fpmaps/default http://api.talis.com/stores/mystore/indexes/metaboxIndex/fpmaps/fpmap http://api.talis.com/stores/mystore/indexes/m21Advanced/fpmaps/fpmap http://api.talis.com/stores/mystore/config/fpmaps/1
If none of those work, please contact us and we'll look it up for you.
[edit] How do I view my store's field/predicate map as RDF?
Append ?output=rdf to the URI or supply application/rdf+xml in the HTTP accept header that you send with your request.
For example:
curl -v -H accept:application/rdf+xml http://api.talis.com/stores/mystore/indexes/default/fpmaps/default
[edit] How do I clear all data out of a store?
You need to POST a Reset Data Job to your store's Job Queue (/jobs). You must have the configure-store capability to perform this operation.
The format of a job request is as follows:
<rdf:RDF
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"
xmlns:bf="http://schemas.talis.com/2006/bigfoot/configuration#" >
<rdf:Description rdf:about="my:jobrequest">
<bf:jobType rdf:resource="http://schemas.talis.com/2006/bigfoot/configuration#ResetDataJob"/>
<bf:startTime>2007-05-02T14:14:00Z</bf:startTime>
<rdfs:label>My Reset Data Job</rdfs:label>
<rdf:type rdf:resource="http://schemas.talis.com/2006/bigfoot/configuration#JobRequest"/>
</rdf:Description>
</rdf:RDF>
Enter the correct time you want to run the job. The time should be in Universal Time (the same as GMT). So if we are in British Summer time, move the time back 1 hour. i.e You want the job to run at 14:00 BST, set it to run at 13:00.
Submit the job using the following curl command:
curl -v -d @job-request.rdf -H content-type:application/rdf+xml http://api.talis.com/stores/mystore/jobs
[edit] My job hasn't run - why?
The most common cause of a job not running is that you've submitted it with the wrong date and time. Double check the date you used and remember that the Platform always runs in UTC which is an hour behind British Summer Time (i.e. 4pm BST is 3pm UTC)
[edit] How can I reindex my RDF?
Submit a Reindex Job to your store's job queue.
