SharePoint MetaData and Classification

Using Medata Properties to Drive SharePoint Search

Because our SharePoint products depend on the use of SharePoint metadata (custom column), we often get asked by customers how they can use Sharepoint metadata in Search.  Most customers would like to be able to do both full text search and also be able to search based on metadata.  For example, if they have defined a metadata column called Project Number, and they want to find all documents related to a particular Project, they should be able to specify the required Project Number in the Advanced Search page in SharePoint. The Advanced Search page does have a property picker that can be be used in search. Unfortunately, by default, the properties listed in the property picker are just some standard SharePoint properties.

Also, customers want to make sure that people only see documents in the Search Results page to which they have permission.  In other words, if you don't have permission to the document you should not see it in a search result. In this post I'll dicuss the first part of this request which is how to use custom metadata in Search.

I decided to set this up on my own SharePoint machine to see how this works.  My impression after having played a while is that this setup is way more complicated than it should be.  There are two parts to this setup:

1) Create a Managed Property which can be used in the search properties.

2) Modify the XML of the Advanced Search web part in order to include your custom metadata as a selectable search property.  

Custom metadata is collected by the crawler when it indexes the contents of your SharePoint site. But you must also expose this custom metadata to the users who perform search queries in the Search Center. This procedure assumes the Search functionality is already configured for your Site.

Create managed properties

Use the following procedure to create managed properties.

1. Go to SharePoint Central Administration.  Select Application Management, and in the left hand bar Under Shared Services Administration click on your Shared Services website (e.g. SharedServices1)

2. On the Shared Services Administration page, in the Search section, click Search settings.

3. On the Configure Search Settings page, in the Crawl Settings section, click Metadata property mappings. Check to see if your property already exisits.  if so, you are good to go. if not, proceed to step 4.

4. On the Metadata Property Mappings page, click New Managed Property.

5. On the New Managed Property page, in the Name and type section, in the Property name box, type the name of the managed property you want to create.

6. In the Description box, type a description for this managed property.

7. Under The type of information in this property, select a property type.

8. In the Mappings to crawled properties section, select one of the following:

   • Include values from all crawled properties mapped. Select this option if you want values from all crawled properties to be mapped. A query for a property in a document in which all crawled properties are mapped returns a result if any of the crawled properties that are mapped match the query.

   • Include values from a single crawled property based on the order specified. Select this option if you want only a single value mapped. When multiple crawled properties are mapped to a managed property, the one that is chosen will be the first in the list that has a value for a given document. You can reorder the list by using the Move up and Move down buttons.

9. If you selected Include values from all crawled properties mapped, skip to step 12.

10. Click Add Mapping to add a mapping to the list.

11. The Crawled property selection dialog box appears. Configure the settings as follows:

    1. On the Select a category menu, click either All categories or a specific type of document category (for example, Office or SharePoint).

    2. In Select a crawled property, select a crawled property to map to the managed property that you are adding.

       Because the list of crawled properties is likely to be long, you can type the name (or the first part of the name) of the property that you are looking for in the Crawled property name box and then click Find.

    3. Click OK.

12. Repeat steps 9 through 10 for each additional crawled property that you want to map to this managed property.

13. On the New Managed Property page, in the Use in scopes section, select the Allow this property to be used in scopes check box if you want this managed property to be available for defining scopes.

14. Click OK.

Once this is done we can now proceed to step #2 which is to modify the Advanced Search page so users will see the property. Here is a screen shot with the Advanced Search page,  at the bottom you can see that you can search based on properties. (Click to see larger view)

But by default only the standard SharePoint properties are available in the search as can be seen in this screen shot.

In my case I wanted to add a property called "Department". This custom column in my Document Library lists what department created the document. In order to do this I needed to modify the XML of the Advanced Search Web Part.  To do this.

1. Go to the Advanced Search Web page.
2. From the Site Actions menu select Edit Page.
3. Select "Modify Shared Web part" from the Edit menu on the search web part.
4. Expand the Properties section and then click on the XML in the Properties box. You should be able to expand the view of the XML by clicking on the ... button.
5. Add the property you want to add to both the PropertDef and PropertyRef sections of the XML.

This screen shot shows how I added Department to the PropertyDef section 

Once this is done and the web part is saved, you should be able to go back to the Advanced Search page and see your property in the list of properties available for search.  Voila! now you can search based on metadata in your custom column.

Of course, metadata can be used for more than just search.  If you are interested in using metadata for security have a look at our metadata security blog.

 

Timestamping and Tracking SharePoint Documents

A number of our customers want to track what happens to their SharePoint documents when they are downloaded or opened.  This is usually because the SharePoint documents contain some sensitive information and they want to make sure their users are handling this information correctly. If documents are being distributed to people who shouldn't have them, or are being printed and distributed inadvertently, organizations would like to know the source of the document. We can support some timestamping and tracking in our current release of Document Marking for SharePoint, and we will do even more in our new release which is scheduled for late February 2010.

Timestamping can track when the document was added to SharePoint, when it was last modified, or when it was opened / downloaded from SharePoint.  In the current release of our product we support the first two, when it was added and when it was modified.  The next release will also support when it was opened / downloaded. In the example below we have timestamped when the document was added or last changed in SharePoint. The timestamp has been applied to the top left header area of the document. (click image to get a larger view)

Another way of tracking documents is by inserting the author, or the name of the user that has downloaded / viewed the document into the document header / footer / or watermark.  Currently we support adding the author, or Modified By name into the document.  In the next release we will support adding the current user's name into the document.  In the example below we have watermarked the name of the user that last modified the document. In this case it was the administrator as I was signed in as the administrator.

These documents can also be converted to PDF format in the SharePoint library and users can be limited to viewing of the PDF version instead of the original Word version.  In this case, the original Word document would not be modified and only the PDF would be marked with the header and watermark. This will be  supported in our next release with the addition of our PDF Control product.

If you have any other requirements for timestamping or tracking documents in SharePoint, let us know.  We would be happy to consider them for future releases of our products.  
 

Balancing Information Security and Information Sharing

We do a lot of work in the US Federal government, and we find that one of the biggest challenges facing the government is their ability to balance information security and information sharing.  The recent attempt to blow up a jet over Detroit has once again highlighted issues of the intelligence agencies not sharing sensitive information with other federal government agencies.

 

We’ve just published a new white paper on how information classification and marking solutions can help government agencies with the challenge of information sharing. The white paper is called “Secure Information Sharing: Balancing the Need to Share With the Need to Protect”.  

 

Information sharing is a critical requirement for achieving government transparency and protecting national security. But there needs to be a balance between information sharing and information protection – especially with sensitive information related to privacy, security, and law enforcement.

To enable effective and secure information sharing, government agencies first need to identify the sensitivity of the information they handle. However, without a tool to help users identify and mark information, markings are often absent, incorrect, or inconsistent. This hinders information sharing and increases the risk of inadvertent disclosure.  This is especially important in environments like SharePoint which are usually put in place to facilitate information sharing.

 

Fortunately, information marking solutions enable users to apply uniform, consistent, and comprehensive markings to emails and documents – with minimal training and impact to regular business processes. Properly marked information helps ensure that the right information reaches the right people, and reduces the tendency to over-protect information. This ultimately improves government transparency and national security.

The white paper examines the challenges of sharing information in today’s government agencies, and introduces the many advantages of implementing an enterprise information marking solution.

 

Here is the link to the full whitepaper if you are interested.  

I hope you find these resources useful, and we welcome any feedback!

-Lara

Controlled Unclassified Information Task Force Releases Report

I noticed the Presidential Interagency Task Force on Controlled Unclassified Information (CUI), led by Secretary Napolitano released its report in mid-December (PDF version of report).  Because the report deals with unclassified information, the recommendations could have a major impact in the way information is classifed and shared in the US federal government.  To this point almost all of the emphasis has been placed on classifying secret information.  The unclassified information has typically been created and circulated with little control. These recommendations would have impact on systems like SharePoint which are normally put in place to share information.   The overall goal of CUI is to implement reforms to enhance information sharing among federal, state, local law enforcement agencies and safeguard sensitive information used by the government.

The report recommends a single, standardized framework for marking, safeguarding and disseminating sensitive but unclassified (SBU) information across the federal government. SBU information refers collectively to the various designations for documents and information that are sufficiently sensitive to warrant some level of protection but that do not meet the standards for classified (secret) information.

If adopted, the recommendations will require the marking of all controlled unclassified information. These markings will appear as header / footers etc in documents and email, as well as other types of information.  The markings also have to be captured as metadata within the documents to ensure gateways and other security systems can read the classification and take the appropriate action (block, let pass, quarantine etc).  The classifications should also show up in SharePoint as metadata in columns. This lets the user quickly know what type of information they are dealing with, and how it can be shared (or not shared).

Security of various documents in SharePoint could also be impacted.  Sensitive but unclassified information will have to be secured in different way from standard unclassified information.  This could mean additional separation via document libraries, or enhanced item level security based on the classification of the documents using software like our Metadata Security for SharePoint solution. Automatic marking of documents with the appropriate CUI markings can be facilitated with our Document Marking for SharePoint solution.

We'll have to wait and see if the recommendations will be adopted as expected in the next few months.  If so, departments will need to start their CUI planning, both technical and budgeting, in order to position themselves and their users for full adoption by the deadline dates.

Charlie....