WinFS

WinFS Beta 1 logo

WinFS (short for Windows File System or Windows Future Storage) was a cancelled storage subsystem that allowed users to search for items based on their metadata, regardless of the item's type of file and the application in which it was created. This in turn would make searching for content independent of format easier.

WinFS was billed as one of the pillars of Windows Longhorn. However, Microsoft announced on 27 August 2004 that WinFS would be in beta testing when Longhorn becomes available, and would be delivered after the Longhorn release. On 29 August 2005, Microsoft (through Vijau Bangaru, a Program Manager on the WinFS product team) announced the release of WinFS Beta 1 for Windows XP Service Pack 2. Regular beta and CTP releases were planned. However, on 23 June 2006, Microsoft (through Quentin Clark, who ran the Program Management Team for WinFS) announced that there would no longer be a separate WinFS release. Instead, some of its parts were included in ADO.NET and Microsoft SQL Server.

WinFS was based on SQL Server 2005, ran on top of NTFS, and used metadata of files to improve organizing, searching and sharing of information. It also stored structured data such as contacts and calendars. WinFS replaced the traditional file system hierarchy of files and folders with items (either based on files or not, such as contacts and e-mails) and associations (relationships between items).

Development

Screenshot of Microsoft Cairo/4.0.1175.1. Note that local disk D:\ is under an OFS partition

WinFS is ultimately traceable to the Object File System (OFS), an integrated store, which Microsoft planned to be part of Microsoft Cairo. The idea behind OFS "was to make sense out of all the bytes of data in the filesystem in a human readable and searchable way, organized as a collection of objects with properties all part of a strongly typed schema." OFS was a relational file system, "which would help manage local and distributed objects using an active storage paradigm" and "would have removed the need to understand where data resided", both locally and remotely. One build of Cairo, 1175.1, allowed OFS partitions (see screenshot). However, it is not clear if the features described for OFS are present. Instead, OFS, as implemented in build 1175.1, appeared to be a predecessor of the Content Indexer (now the Indexing Service) and Active Directory.

Cairo fell apart in the mid-1990s. OFS merged into Storage+, which Paul Thurrott wrote, Microsoft would turn its attention to after Windows 2000 ships. Storage+ would replace the NTFS file system and with a relational, object-oriented file system based on SQL Server 8.0, turning the file system into a relational database, speeding searches and providing a higher level of granularity for those searches Storage+ later became the Relational File System, which Microsoft intended to but failed to ship with SQL Server 2000.

Storage+ (at this point "a new SQL Server .NET-based file system") was then slated for Blackcomb, but by 2002, Thurrott verified that it would ship with Longhorn. By 2003, Storage+ was called WinFS, would include technology from SQL Server "Yukon", and would not be a file system but a service on top of NTFS.

During PDC 2003, Quentin Clark, director of program management, WinFS, explained:

WinFS is...

• All end-to-end data lives in Longhorn
• New user experience in the Longhorn shell
• A trustworthy place to store data
• Data model built on relational database technology
• File system capabilities built on NTFS
• Everyday info - domain specific schemas - Longhorn will come with set of schemas Services that make data interactive

WinFS is not...

• Abandoning legacy applications
• Forcing the world to rewrite everything
• Complete yet
  • PDC is good preview of model, feedback welcome
  • Fundamental model complete for Beta 1
  • Platform and UE complete in Beta 2
• Just a database - MS has set of relational assets, WinFS is pulling techs from NTFS, SP, and CLR too

Longhorn build 4051, which was distributed during PDC 2003, contained WinFS. However, it suffered from huge performance issues. However, on 27 August 2004, Microsoft announced that WinFS would be in beta testing when Longhorn becomes available, and would be delivered after the Longhorn release. This meant that WinFS would be removed from Longhorn and would ship at a later date.

On 29 August 2005, Microsoft quietly made WinFS Beta 1 available to MSDN subscribers, but announced it officially through the WinFS Team Blog. Beta 1 required Windows XP Service Pack 2, and .NET Framework 2.0.50215.322. The WinFS API was included in the System.Storage namespace.

During PDC 2005, Microsoft released a video presentation, named "IWish", showing how applications would advantage of a unified type system. The concepts shown in the video ranged from applications using the relationships of items to dynamically offer filtering options to applications grouping multiple related data types and rendering them in a unified presentation. Steve De Mar, the Product Designer for the WinFS team, explained the goals of IWish:

Our goal was to peak the curiosity of you, the developers, to get you thinking about what could really be built on WinFS. We wanted you to see how WinFS enables a totally new class of user experiences. The UIs in the video are all based on some of the great prototype applications that have already been developed for the WinFS platform. Although we cast them in a more futuristic visual “theme” to fit the style of the video, the functionality we showed exists in the prototypes that either we’ve developed in house or we’ve seen others develop on WinFS. A number of these were demo’d at the PDC, and we've more on the way which you should be hearing about shortly.

We hope the video inspires you to bring to life cool new innovative applications for WinFS!

In addition, during PDC 2005, Shishir Mehrota of WinFS Product Planning gave an overview of WinFS, Program Manager Ramesh Nagarajan gave a presentation on "“WinFS” Future Directions: Building Data-Centric Applications Using Windows Presentation Foundation “Avalon” and Windows Forms", and Samuel Drucker, Development Manager, gave a presentation on WinFS and ADO.NET (on how ADO.NET is evolving to incorporate some of the WinFS aspects) and WinFS Programming with LINQ.

Beta 1 Refresh was released to MSDN subscribers on 1 December 2005 to be compatible with the final release of .NET Framework 2.0.

WinFS Beta 2 was planned for some time later in 2006. Four WinFS sessions were planned for Tech Ed 2006. Beta 2 would have been integrated with Windows Desktop Search, so that the latter could display both WinFS and non-WinFS files.

But on 23 June 2006, Quentin Clark announced through the WinFS Team Blog, "With most of our effort now working towards productizing mature aspects of the WinFS project into SQL and ADO.NET, we do not need to deliver a separate WinFS offering."

Many of the principal features Microsoft intended to provide with WinFS included a pane for metadata property editing, breadcrumb-based property navigation, filtering or stacking items over properties, incremental search, and saved searches; features were incorporated in Windows Vista. Query composition, a WinFS feature that allowed users to perform additional searches that reuse the results of a previous query, was later incorporated in Windows Vista.

Examples of uses of the technology are the object-relational mapping components into ADO.NET Entity Framework; support for unstructured data, adminless mode of operation, support for file system objects via the FILESTREAM data type, and hierarchical data in SQL Server 2008, then codenamed Katmai, as well as integration with Win32 APIs and Windows Shell and support for traversal of hierarchies by traversing relationships into later releases of Microsoft SQL Server and the synchronization components into Microsoft Sync Framework.

In 2013, Bill Gates cited WinFS as his greatest disappointment at Microsoft:

We had a rich database as the client/cloud store that was part of a Windows release that was before its time. This is an idea that will [re-emerge] since your cloud store will be rich with schema rather than just a bunch of files and the client will be a partial replica of it with rich schema understanding.

Limitations of traditional file systems

Traditional file systems, including NTFS which is used in modern versions of Microsoft Windows, store data only as a stream of bytes packed in a file. The Microsoft WinFS SDK Beta 1 Documentation explained the limitations of this approach:

In a file system, data is stored as one or more byte streams packed in a file. This flat data structure does not make the information easily identifiable. The only viable way to identify it is to use the file name and its directory path, that is, a folder hierarchy. Although a file may contain many kinds of information, it is the smallest unit of data that can be moved around and operated on. For example, to read a paragraph of a document, you need to retrieve the entire file and open it in a document viewer.

As Richard Grimes explained:

The perennial problem with data is that you always have too much of it and it is always too difficult to find what you want. This problem will get far worse in the near future when typical machines will have hard disks measured in terabytes. The bigger the hard disk, the more data you will want to store, and the more difficult it will be to find a particular item. Finding a piece of data depends on identifying the specific item, which means that the item has to be categorized.

For example, imagine that you have several thousand digital photographs. Typically, your camera and desktop software will catalog this data by naming the photos in sequence and maybe place these in a folder according to the date that the photos were taken. Searching for a particular photo in this scheme involves remembering the date and photo number. If you do not know either of these things, you will have to resort to looking through thumbnails of the photos. Of course, if you have the patience, you could name each photograph as you download it to your desktop. A coarse-grained system, like the file and folder system in Win32®, is not flexible enough to allow you to store all the data you need.

Desktop software may fill this gap, allowing you to build a catalog that stores other descriptive properties like camera settings, light condition, and a general description of each picture. Such software, of course, is an implementation of a database, but the important point is that it acts as a supplement to the file system. You could argue that these descriptions of your photos—metadata—are actually part of the photograph's object. Digital photographs hold other information within the image file's EXIF (Exchangeable Image File Format) header. This header contains information obtained when the photo was taken. To include this metadata in a search would require a query engine to extract this information from each candidate photo and compare it with the query condition. A sophisticated photograph organizer could extract the EXIF information when the photograph is transferred to the desktop and include it as additional metadata in the database.

Desktop search allowed a user to see different types of data from different sources. However, these data still resided on different sources, different places.

Grimes pointed out that one of the monumental problems organizations face is aggregating information that is stored in disparate formats. Knowledge workers have long wanted to be able to search for content independent of format. Sean Grimaldi of the WinFS team called the existence of many formats with different expectations and behaviors a "divide-and-conquer approach" in dealing with data. Grimaldi extensively discussed the negative effects of this approach and summarizes:

The data is divided into file, database, and other data. The negatives are that programmers have to recreate data types, bridge between data types, and rewrite behaviors repeatedly. If this seems normal to you, it may be because you have operated in this fractured data environment for years and have come to expect nothing more.

The Microsoft WinFS SDK Beta 1 Documentation described the existence of proprietary file formats as "separate islands":

Proprietary file formats tend to aggregate data into separate islands, creating barriers for data sharing between applications. For example, an address book created by an e-mail system, such as Microsoft Outlook, is not directly usable by an appointment-setting application unless the latter adopts the same file format or knows how to translate the format from one to another.

Shawn Wildermuth pointed out what this system means for developers:

Data integration solutions must first try to make sense of all the different places applications can store information. Let's look at a typical example:

I am writing an application, I need access to the user's Contact List, Documents folder, and Music Folder. To access each of these data stores, I have to deal with three different APIs to get that data (MAPI to access Contacts, System.IO for the Documents folder, and the Windows Media Player SDK to find all the music for a user). It would be better if we had a consistent API for users' data.

But that is not the worst part of the problem. If my application wants to tie its data and these external pieces together, it will need code that not only establishes the relationship between the objects, but deals with changes to that data and handles data synchronization problems such as orphaned data.

Using common file formats is a workaround to this problem but not a universal solution; there is no guarantee that all applications will use the format. Other solutions used by corporations are database systems and XML and XML web services, as Thomas Rizzo, director in the Microsoft SQL Server group, explained:

In the beginning, computers could not handle the complexities of modeling and automating many business processes. However, with faster computers, better programming languages, and better data technologies, automation of business processes became a way for corporations to streamline their operations, especially with the advent of enterprise resource planning (ERP), customer relationship management (CRM), sales force automation (SFA), and other types of enterprise applications. With these new types of applications, data suddenly became more complex—but at the same time, data had better structure that was more useful for businesses. Rather than storing opaque binary data or simplistic data models, database systems could relate different types of complex data together. For example, ERP systems understand and can model the complexities of general ledgers, human resources, and sales systems. Since standardized schemas described the data in the system, corporations could ask interesting questions about their business and query their systems for that information.

However, working across systems is still a problem in the industry. Not only is it hard to integrate systems within a corporation, but integrating systems across corporations is even harder. This has sparked the growth of enterprise application integration (EAI) vendors who help customers navigate the sea of schematized data across heterogeneous systems. Recognizing this integration problem, the industry has looked to XML and XML Web services as a way to help companies work across multiple systems and multiple organizations. These newer standards help solve data integration problems and are helping to democratize data throughout and across organizations.

Traditional file systems also provide only a single way of organizing the files, namely via directories and file names. Vijay Bangaru of the WinFS team illustrated the problems with this:

If you’re like me, you probably have tons of digital pictures on your computer. My photos are currently organized in a simple hierarchy. In “My Pictures”, I have a bunch of folders that are named something like, “2004-11 Thanksgiving”. Within this folder are all my photos from that holiday. I’m sure you can already see the limitations in this structure.

My latest challenge in actually using my pictures came the other day. My grandparents wanted me to send them some pictures of my fiancée and me. Wouldn’t it be really cool if I could just tell my computer, “Show me all the pictures that have Crissy and me in them”. Instead, I had to scan through my folders to see what events we were both at and copy all the pictures with both of us in them to a temp folder. Then I needed to go through the temp folder to pick which ones to send. Of course, I didn’t remember to delete the temp folder; so it’s eating up drive space. Good thing hard drives are so cheap nowadays.

A traditional file system can retrieve and search data based only on the directory path:

If the user knows the directory path to the file, searching is simple. But if she does not know where the file is, she must walk down from the root directory of a file system—for example, by selecting Search from the Start menu—to traverse the available directory paths until she finds the file. When there are many files on a computer, the search can take a long time. In the NTFS file system and other traditional file systems, the directory path is more or less the only way to identify a file. But users tend to think of data more naturally in terms of its types and attributes—that is, the structure or shape of the data: a document or an e-mail message, its title or subject, the author or sender. How the data is stored is generally of little or no interest. Therefore, reliance on paths is an inherent limitation of the tree-based structure of traditional file systems.

A user may put a file into multiple directories, but this wastes disk space. Also, when a copy is changed, the other copies no longer represent the same data as the changed copy. Hard links that can be created to represent a file by multiple paths are also difficult to manage: links will break when a file is renamed.

WinFS as solution

The core concept of WinFS was that by storing data in a cohesive data store, not just a file system, it would have allowed systems and users to store rich metadata about a myriad of objects and to relate these objects. Compared to a file in a traditional file system, a "WinFS" item would be the smallest unit of data that could be moved around and operated on by different users or applications.

The Microsoft WinFS SDK Beta 1 Documentation explained that WinFS would have provided comprehensive object-relational mappings to let users view and work with their data as an item on a Windows computer. With items, a user or an application would have found it more logical or natural to view and use data. This should have made organizing and querying data, especially complex data, less burdensome. The documentation provided as an example:

For example, consider an e-mail message containing two photos as attachments. In the tabular form, this is most likely modeled as three rows in two tables. One table stores all the e-mail messages and the other stores all the photos. To get the message, a user or application must query two tables and perform the so-called UNION operations on the tables. In the itemized form, the attachment is part of the item. Querying the e-mail message item will return the attached photos as well.

WinFS would have served all kinds of data, whether structured (as in tables/rows or items), semi-structured (as in an XML file), or unstructured (as in a flat byte stream).

The Microsoft WinFS SDK Beta 1 Documentation and the IWish video summarized the benefits of WinFS integration into four points: Unify, Organize, Explore, and Innovate.

Unify

WinFS would have allowed data sharing between applications, as Grimaldi explained:

Because applications can build on WinFS-defined data types, applications can share data—assuming the correct permissions. This is good for you as a programmer, as well as for the end user of your application.

As a programmer, you can reuse data the end user already has in the WinFS store. For example, assume your application presents advertisements to a user of events they could attend, like a bike race or a rodeo.

Before presenting an event to the user, your application can check the user's calendar and display the event differently if the user already has a meeting scheduled that would conflict with the event, such as making the border of the invitation red.

You can also populate data into the WinFS store to be used by other programs. For example, if the end user accepts the invitation to the bicycle race, their personal information manager (PIM) application, such as their e-mail client, could show that date as reserved in their calendar. This also gives the end user a rich user experience in both applications.

WinFS would have placed data from multiple applications into one central store. Bangaru showed the advantages of this:

With WinFS, I can move all my contact information into one central store. My applications can interact with this information and I can synchronize it wherever I want to. The obvious benefit is that I can go to one place reliably for the most up to date information. Now, I could easily add whoever I wanted to an Evite for a Halloween party. No more need for typing in email addresses or importing all my contacts (into yet another silo). When I order flowers for mom on her birthday, it’s just one click for me to fill out the information for the recipient.

In addition to being a central data store, WinFS can be your synchronization hub. I can sync between WinFS Stores and I can sync changes back and forth with legacy stores (like Outlook, PDAs, and cell phones). Now, I can have my latest contact info on my phone, make changes on the road, and sync that information back to WinFS. This, of course, enables some great offline experiences. When I’m doing work on vacation, I don’t need to worry about manually synchronizing my data with my desktop machine once I’m home. Now all my applications will behave like Outlook 11’s awesome offline mode.

Of course, this applies to all data; not just contacts. Now, all my calendar apps will show my Evite invites. My photos are now associated with contacts and events, so I can easily say, “Mail these photos to everyone that was at the event”. When you start adding email, RSS feeds, documents, music and everything else to this, it can get pretty exciting.

Organize

Bangaru illustrated how data could be related, and how related data could be shown:

With WinFS, I can relate my contacts to my photos, so I can easily query for all the pictures that are related to a set a contacts. A photo app built on WinFS could display all the pictures with Crissy and me in them. Then, I could select the good ones to email or print. If you’ve been playing with the Beta 1 bits, you know how easy it is to write an app with this behavior.

Plus, I can now easily check to see what photos I’ve already sent, so I don’t send the same ones again. How? Tell the computer to “show me the photos with both of us that I’ve sent to my grandparents.” All these containment relationships and associations would already be in my store.

Again, this applies to much more than pictures. I can associate a document with a meeting, or music and contacts to a party invite. And these aren’t keywords added to a file, these are associations between real items. All of these organization constructs are just waiting to be exploited by a WinFS app.

Also, all these types are extensible. So for my wedding guest list, I can extend the Contact type to have properties for “Sent ‘Save the Date’ card”, “Sent invitation”, “RSVP Received”, “# of Guests”, “Vegetarian meal?” and so on. That could make things significantly easier. Good thing I can do this with the Beta bits. If we had to wait for WinFS RTM to plan the wedding, I think Crissy would make sure that I wouldn’t be able to blog anymore. :-)

Explore

WinFS natively recognized different types of data by defining common item types, such as audio record, meeting, contact, person, document, photo, message, video clip. This would have solved the problem of querying the following data:

You could write code that shows a picture of a person, his role, and his contact information for everyone attending a given meeting and every document that they contributed to that you have not already read, and on a personal note, if they have already given you some money for the charity bicycle race you are participating in next month. Today, I have all this data available to my computer, but no feasible way to query it without WinFS. I plainly can't get at a lot of my data.

WinFS would have allowed queries such as "Show me all email from people I am meeting with this week," that NTFS could not.

Innovate

Bangaru explained how WinFS would have done away with proprietary file formats:

WinFS is built on a true relational store that all applications can leverage. Today, tons of popular apps lock their data into proprietary stores. This includes photo apps like Photoshop album, music apps like WMP and iTunes, financial apps like Quicken and Money, email apps like Outlook, and so on. By using WinFS, developers can concentrate on building rich applications without worrying about creating a storage silo. After all, WinFS can give applications a place for all their structured, semi-structured, and unstructured data.

In addition, WinFS will provide services like synchronization, notifications, rules, and backup/restore. The services provided by WinFS lead to a zero-admin and extremely reliable experience.

Synchronization was explained earlier under Unify.

Kati Dimitrova, a developer on the WinFS API team, explained how WinFS Rules work through a scenario involving Mary, a recruiter trying to manage and track candidates throughout her day across multiple applications:

First and foremost, WinFS rules are persistable units of logic. In this scenario, Mary creates the query “candidates that are interviewing today” through her WinFS-enabled recruiting application and persists it. But how is the query persisted? As a WinFS item. In fact, all WinFS rules, including queries, are items. This has many advantages. To start with, rules can be created, manipulated, stored and secured in the same manner as all other items.

Further, WinFS rules are about discovery and sharing of persisted logic among applications. Because rules are WinFS items, they can also be queried for, discovered and shared among applications just like all other WinFS items. In this scenario, Mary can easily find and use the query “candidates that are interviewing today” from her email application even though she created it through her recruiting application.

Finally, WinFS rules are about re-usability and composition. Not only can Mary find the query she created in her recruiting application, she can re-use it in her email application to create a new query “mail related to candidates that are interviewing today” that refers to the original query. Note that composition of rules is true composition; if Mary changes the logic that defines the query “candidates that are interviewing today,” those changes will affect the results of the query “mail related to candidates that are interviewing today.” Bangaru explained two levels of backup that WinFS would have included. Item level backup would have allowed a user to revert a change to an item. Store level backup would have covered the situation of a broken hard drive and a user needing all his files back.

Data storage

Architecture

What is WinFS?

This section discusses WinFS as it worked under WinFS Beta 1, the latest release of WinFS to the public, not under any Longhorn pre-reset build. All documentation therefore should refer to the WinFS Beta 1 time frame, PDC 2005 included.

WinFS used a subset of SQL Server 2005 technology, modified for file storage. It consisted of the following components:

• "WinFS" Core - the core services expected from a file system, and included the relational storage engine;
• Data model - the technical innovations of WinFS, including basic item structure, relationships, and the ability to extend both items and relationships;
• Schemas - built-in schemas allowed WinFS to understand a user's data in richer ways than traditional file systems and provide a more structured way to handle metadata. Included were first, schemas for everyday information such as documents, e-mail, appointments, tasks, media, audio and video, and second, system schemas that define configuration, programs, and other system-related data;
• Services - included syncronization and rules (discussed below);
• APIs - which developers might have used to program the different building blocks of the "WinFS" system, including data operations, rules, synchronization, and the data model

WinFS required three services, which ran in three processes:

1. WinFS.exe, the WinFS data store, that ran a Yukon (SQL Server 2005)-derived database engine
2. WinFSSearch.exe, which ran the WinFS search engine
3. WinFPM.exe, which ran the WinFS File Promotion Manager. File promotion referred to the process of extracting data from a file and populating the data to an item. The reverse process of persisting itemized data to streams of the underlying file was called demotion. WinFS served all kinds of data: structured, semi-structured, or unstructured, through a new file system driver (RsFxDrv.sys). This driver allowed a user to access a WinFS item both as a file stream (as a file) through the Win32 API or as an item via the WinFS API. The process of file promotion and demotion kept the file-based and item-based representations of a WinFS item synced. This made WinFS backwards-compatible with NTFS.

WinFS depended on the .NET Framework, and used the same APIs as other data types thanks to ADO.NET. The WinFS API was included in the System.Storage namespace.

Data store

WinFS data hierarchy

WinFS kept data in Stores, which in turn are kept in a relational backend. A WinFS store was a common repository where any application can store items and associations. Items could either be backed by files (for example, images and documents) or not (for example, contacts and e-mail). Associations established relationships between different types of data (items), which then allowed users to organize information based on how they used them. Bagnaru explained how items and associations were stored:

Q: Mikael writes, “How and where are all the relations, keywords, etc. saved?”

A: The item properties are stored in the WinFS store as part of the item itself. WinFS has a concept of “links” which are an entity type with a source item and a target item that relate items. There is also the concept of “common value” relationships which are based on item properties. Navigation of common value relationships is a value based join.

All data in the store were accessible as CLR objects, allowing a developer to leverage all their Visual Studio and .NET knowledge while building apps..

WinFS, by default, came with two stores, DefaultStore and Catalog. DefaultStore was the default location for user and application "WinFS" data. Catalog stored metadata about "WinFS" stores and shares, as well as other system data. The Catalog store was read-only and should not be used for user data. Users could create additional stores, but a store cannot span multiple volumes and could only be hosted on NTFS volumes. WinFS stores could be accessed through a top-level object in "My Computer" in Windows XP, or as a share in "My Network Places". This is because every WinFS store had a root share with the same name as the store.

WinFS stores were made up of database files stored in a directory under the "System Volume Information" folder. Filestreams were saved as ordinary NTFS files with a GUID as file name. Meanwhile, metadata is stored in a SQL Server database.

Data model

The WinFS data model is the architecture of the type system. WinFS introduced a type definition language, an XML vocabulary but not the XML Type Definition Language, to describe types.

WinFS provided a set of schemas that defined types and attributes for implementation by the WinFS API. A developer could also create custom schemas.

A schema was a collection of type declarations, such as the declaration of a postal address, a full name, or a person. Each schema defined a set of types that mapped to some information. All these types derived from a set of base types defined by "WinFS". The base types grouped related types together, creating type families.

The root types in the WinFS system were Item, Link, InlineType, and ItemFragment. The other WinFS types derived from the root types. Descendants of one of these types were entity types. For example, Person was derived from the Contact type, which derived from the Item type. AttachmentLink was derived from the Link type. Therefore, Person, Contact, and AttachmentLink were all entity types. An EntityType had its own unique identity, and so could exist on their own. In contrast, an InlineType did not have a unique identity and could exist on its own. FullName of a Person is an InlineType, a Person is an EntityType.

ItemFragments were also entity types, which enabled the declaration of large collections in the item types, such as multivalued properties. Each collection member had a key, which enabled applications to access individual members of the collection independently of the owning item. An example would be a message, which might have a large number of participants. The ItemFragment would be participants under the Item message.

Meanwhile, an EntityExtension allowed a user to extend built-in items by adding further properties as necessary. WinFS schemas defined several extension types. For example, the System.Storage schema namespace defined two ItemExtension types: Reminder (such as a birth date Reminder to a Person) and ItemKeywords (which allowed a user to assign keywords, such as "legal document" or "personal document" to a Document).

Links and associations

Links referred to data that defined links between two or more distinct existing item types; for example, an Item type, Message, might have attachments that were documents, another Item type. WinFS also provided the concept of Associations to relate items. Associations were not physically stored in the store. An example of an Association would be that declared by a user between the Document and Contact types when identifying the publisher of a document.

WinFS depended on applications and users to provide the information to create links. According to Bagnaru,

Q: Mikael also asks, “Do I have to set all relations manually? How does WinFS know which document is created by which user?”

A: The relationship “setting” will depend on how it is modeled. A link will need to be set by something (i.e. metadata handler, sync adapter, application, etc.) A common value relationship, however, is just based on a join. WinFS depends on the application and user to provide the information to create links. In many cases, an application already has this data; for example, Office applications already have an “Author” property. In other cases, the user already knows this information and can set it. It would be fairly easy to get my primary contacts’ information moved into WinFS and then relate pictures to the contacts. In the future there could be many processes that could be used to relate your data: natural language processing of documents, concept extraction, face recognition for images, etc.

Item

The Item was the fundamental concept in "WinFS". An item was the smallest data element defined by WinFS, and was the equivalent of an object in object-oriented programming. Each item had a unique identifier, its GUID.

WinFS defined common item types, which resided in the following WinFS namespaces:

• System.Storage.Audio
• System.Storage.Calendar
• System.Storage.Contacts
• System.Storage.Documents
• System.Storage.Image
• System.Storage.Media
• System.Storage.Messages
• System.Storage.Video

An item that would fall under System.Storage.Documents would lead to, as Malik explained:

For instance, if you author a Microsoft Word document, it is automatically promoted to an instance of System.Storage.Documents.Document, which then automatically populates its various relevant properties such as title, subtitle, author, etc.

If a file did not fall under any of the common item types, WinFS stored them as a GenericFile item. This happened when no file metadata handler was registered for a file name extension. A file metadata handler promoted and demoted properties between structured file formats and "WinFS" items. (See #Architecture for discussion of file promotion and demotion.)

A definition for item types then reduced the amount of code to bridge between types. Grimaldi showed the benefit of this:

For example, if your application and my application both use the WinFS person type, no transformation is necessary. Even if I extend the WinFS person type, we may be able to share the data without transforming it. For example, if I write an extension to the person type that adds a property for their medical insurance member ID, and your application is a greeting card application, I can pass a person between our applications with just a cast to person.

Another benefit is common behaviors. Only one code is necessary to save an item, regardless of type. For example, a document may be saved using the same code below by replacing person with document, and Sean Grimaldi with the display name of the document. // Access the WinFS store. using (WinFSData wd = new WinFSData()) {

   // Create a person.
   Person person = new Person(wd.GetRootItem());
   person.DisplayName = "Sean Grimaldi";
   
   // Save the person.
   wd.SaveChanges();

}

Defining common item types also allowed WinFS to look inside files in a similar manner to desktop search engines. Bagnaru pointed out:

Q: Lars asks, “Does WinFS look inside files (word, excel, pdf, etc.) like desktop search engines do?”

A: Yes. When a file is stored in WinFS, its metadata will be extracted by type specific property handlers. Part of our Beta 2 work is to integrate this with the Windows Desktop Search handlers. The end result is that a Windows Desktop Search query will return WinFS and non-WinFS files.

Access control

Bagnaru explained how WinFS secured data:

Q: Chris asks, “While it's good to have a common silo that all apps can easily use, it's seems a bit risky. Wouldn't some rogue app/spyware have an easier time getting to financial Quicken/Money data?”

A: Even though all your data is in a single silo, all the data isn’t equally accessible. WinFS provides two data protection mechanisms. First, there is share-level security that controls access to your WinFS share. Second, there is item level security that supports NT compatible security descriptors. In order to access an item, the caller must be provisioned at the both share and the item. To manage this, WinFS will provide a rich API for security administration. Also in Vista there is the concept of “integrity level” for an application (Mandatory Integrity Control). Data can be configured to prevent lower integrity apps from accessing it. Simon (our security PM) will post more detailed description about this soon.

Data retrieval

WinFS used OPath, a string-based query language used for filtering, sorting, and projecting information stored in the WinFS store.

Data sharing

WinFS allows easy sharing of data between applications, and among multiple WinFS stores, which may reside on different computers, by copying to and from them. A WinFS item can also be copied to a non-WinFS file system, but unless that data item is put back into the WinFS store, it will not support the advanced services provided by WinFS.

The WinFS API also provides some support for sharing with non-WinFS applications. WinFS exposes a shell object to access WinFS stores. This object maps WinFS items to a virtual folder hierarchy, and can be accessed by any application. WinFS data can also be manually shared using network shares, by sharing the legacy shell object. Non-WinFS file formats can be stored in WinFS stores, using the File Item, provided by WinFS. Importers can be written, to convert specific file formats to WinFS Item types.

In addition, WinFS provides services to automatically synchronize items in two or more WinFS stores, subject to some predefined condition, such as "share only photos" or "share photos that have an associated contact X". The stores may be on different computers. Synchronization is done in a peer-to-peer fashion; there is no central authority. A synchronization can be either manual or automatic or scheduled. During synchronization, WinFS finds the new and modified Items, and updates accordingly. If two or more changes conflict, WinFS can either resort to automatic resolution based on predefined rules, or defer the synchronization for manual resolution. WinFS also updates the schemas, if required.

Application support

Shell namespace

WinFS Beta 1 included a Windows Shell namespace extension, which showed WinFS stores as top level objects in "My Computer". Files could be copied into and out of the stores. Applications could be directly used to save there through the share \\COMPUTERNAME\DefaultStore in "My Network Places". Even folders such as My Documents could be redirected to a WinFS store. WinFS used Importer plug-ins to analyze the files as they were being imported to the store and create proper WinFS schemas and objects, and when taking the objects out, re-pack them into files. If importers for certain files were not installed, they were stored as generic File types.

Microsoft Rave

An application that shipped with WinFS Beta 1, Microsoft Rave allowed synchronization of two or more WinFS stores. A user could synchronize between multiple computers running WinFS, and also between different users

Microsoft Rave supported synchronization in full mesh mode as well as the central hub topology. While synchronizing, Microsoft Rave will determine the changes made to each store since the last sync, and update accordingly. When applying the changes, it also detects if there is any conflict, i.e., the same data has been changed on both stores since the last synchronization. It will either log the conflicting data for later resolution or have it resolved immediately.

Microsoft Rave used peer-to-peer technology to communicate and transfer data.

StoreSpy

File:Storespy.jpg

WinFS included StoreSpy, a WinFS Item browser that could be used to browse structured data instances with their properties and relationships.

With WinFS Beta 1, Microsoft included an unsupported application called StoreSpy, which allowed one to browse WinFS stores by presenting a hierarchical view of WinFS Items. It automatically generated virtual folders based on access permissions, date and other metadata, and presented them in a hierarchical tree view, akin to what traditional folders are presented in. The application generated tabs for different Item types. StoreSpy allowed viewing Items, Relationships, MultiSet, Nested Elements, Extensions and other types in the store along with its full metadata. It also presented a search interface to perform manual searches, and save them as virtual folders. The application also presented a graphical view of WinFS Rules. However, it did not allow editing of Items or their properties, though it was slated for inclusion in a future release. But the WinFS project was cut back before it could materialize.

Type Browser

WinFS also includes another application, named WinFS Type Browser, which can be used to browse the WinFS types, as well as visualize the hierarchical relationship between WinFS types. A WinFS type, both built-in types as well as custom schemas, can be visualized along with all the properties and methods that it supports. It also shows the types that it derives from as well as other types that extend the type schema. However, while it was included with WinFS, it was released as an unsupported tool.

OPather

WinFS Beta 1 also includes an unsupported application, named OPather. It presents a graphical interface for writing Opath queries. It can be used by selecting target object type and specifying the parameters of the query. It also includes Intellisense-like parameter completion feature. It can then be used to perform visualization tasks like binding results of a query to a DataGrid control, create views of the data in WinFS itself, or just extract the query string.

Project "Orange"

Microsoft launched a project to build a data visualization application for WinFS. It was codenamed "Project Orange" and was supposedly built using the Windows Presentation Foundation. It was supposed to provide exploration of Items stored in WinFS stores, and data relationships were supposed to be a prominent part of the navigation model. It was supposed to let people allow organization of the WinFS stores graphically as well – productizing many of the concepts shown in the IWish video (discussed above at #Development). However, since the WinFS project went dark, the status of this project is unknown.

External links

• WinFS Blog
• WinFS at the Microsoft Developer Network (Wayback Machine archive)
• Channel 9 Videos
• WinFS Newsgroup
• WinFS Beta 1 Preview
• WinFS area on NetFXGuide.com (Wayback Machine archive)