Expertise Search in the Enterprise

The good news is that everyone (except for a few executive technophiles who fear workers Twittering the day away) seems to agree that providing tools to employees that allow them to form networks with others in their organization to collaborate and share knowledge is a good idea. The bad news is that there doesn’t appear to be much consensus about which tools are best suited for corporate social networking.

Some argue that since employees are already familiar with easy-to-use consumer sites such as Facebook and MySpace, companies should co-opt them for work. Others argue that there are too many security and privacy concerns with these public sites and that companies should implement social networking software designed specifically for business or even build their own using a hodgepodge of open-source code and internal development.

Unfortunately for the latter camp, a recent CIO magazine article (April, 2008) reports that the adoption of corporate social networking is not going as well as originally expected. Why is this? The article explains, "Social networks for internal collaboration seem like a good idea in principle, but two obstacles are so far inhibiting their adoption: tools to automatically feed business information to the networks, and the challenge of vying for attention with Facebook and MySpace."

It appears as if two of the biggest turn-offs for users of corporate social networking are that they don’t want to have to add a plethora of information into yet another application and that they view their work-version platform as dull in comparison to its exciting consumer cousins.

Fair enough. But what if you brought users to the information first and provided an interface with them to connect with the colleagues who own the information? That interface wouldn’t try to mimic the personal relationships that friends and family have, but instead be designed for work.

What’s needed is an application that can bring together all of the content available in your organization via a search interface that allows interaction with others through that content.

Enterprise Search as Expertise Search

One of the critical differences between consumer social networking versus business social networking is whom you are building networks with and what information you share with them. At work, you’re not looking for people to "friend you," you are searching for someone with the expertise to help with an important project. It is called work and not fun for a reason.

Moreover, you probably don’t really care about what books your colleagues love or what recent movies they’ve seen. But you are likely interested in what research they’ve published and what reports or presentations they’ve authored, tagged or written comments about. This is where enterprise search—specifically social search—comes in.

When organizations use tools such as Vivisimo Velocity social search interface to vote, rate, tag and annotate search results, not only is the tag stored in the search index as information about the document, or metadata, but so is information on who created the tag and when it was created.

For example, when an employee (let’s call her Joyce Reed) tags a search result with the word "mobile," the system not only knows that it was Joyce Reed who created that specific tag on July 2, 2008, it also knows that she is from the marketing team. (See Figure 1, PDF or Page S10, print version.) These details then become part of the metadata and are a potential source of new knowledge and information related to the document, just as the original metadata such as author and publication date are.

Social Search at Work

So how do employees use all of this information stored in the search index to network with their peers? Let’s use the example on the left to illustrate.

First, a user types the word "mobile" into the search box on the interface and hits enter. Next, the search engine will return results matching that query and will also return information about potential experts (i.e. Joyce Reed) regarding the query "mobile." The search index queried can have Joyce’s employee data such as photos, contact details, authorship, biographical profiles and recent tagging activity that it has extracted from multiple data repositories.

Each potential expert’s information is combined into a single search result, delivered to the user by a mash-up that appears at the top of the search interface. With these mash-ups, employees can easily find topical experts within their organization via search. The searcher can then reach out to any experts they find directly using the contact information provided. Alternatively, they could also navigate through their colleague’s tags and other metadata—such as authorship—to understand what other content has been identified as useful by that person without ever picking up the phone or sending an email.

The Future: Personalized Social Search

Conducting a search and looking through user metadata is just one way to find experts, though. In the future, you will have access to not only user metadata, but access to personal profiles that provide a snapshot of an employee and their activities in real time. Above the search results is the most current information about an employee. With just a quick glimpse, you can quickly view a co-worker’s most recently authored documents, their team members and even the last few email exchanges you have had with that person.

By using the navigation elements shown on the left, you can easily identify the file types that someone has published, their top-used tags and see a graph representing when emails have been shared with you. By including email in the search platform’s personal profiles, you begin to create an environment that cannot only connect you with colleagues, but one that is personalized for you.

Leveraging the Power of People

Social networking within the enterprise doesn’t have to be a disappointment, either for you or for your end-users. By not expecting users to enter data into yet one more application and by not trying to duplicate an interface designed to foster interpersonal communications between friends and family, you’re halfway to success.

Deploying an enterprise search platform that can bring all of the information to your users with an easy-to-use interface that allows them to network with one another regarding that information will get you the rest of the way there.

Best Practices of Enterprise Search

An enterprise search platform should not confine organizations to a one size fits all deployment. It is critical in today's environment is tailored, adaptable experience that enables users to access the exact answers they need within the context of their particular business area.

Here are examples how search can be used.

Government 

1. Respond to information requests (FOIA), finding and packaging relevant emails, scanned documents and electronic files quickly. 

2. Find, isolate and remove personally identifiable information (PII) from network shares, and receive notification if more is added.

Human Resources 

3. Increase self-service, enabling employees to easily find answers to specific policy, benefits or other questions from an HR portal. 

4. Sharpen recruiting by pinpointing candidates based on content in their resumes, correspondence, LinkedIn profiles and more.

Education 

5. Empower students to find the latest information on classes, schedules, activities and critical notices using mobile devices. 

6. Provide unified access to research, archives, databases, core facilities management and other content scattered across multiple campus systems.

Financial Services 

7. Compile all documentation surrounding a loan application and easily direct the packet through the approval process. 

8. Uncover relevant investment information from the thousands of emails that staff members are sent every day. 

9. Empower customers to access your research and analysis online, searching through different sources to help them make decisions.

Retail 

10. Review marketing plans, logistical details, financial forecasts and planograms stored in regional SharePoint sites to prepare for an annual product rollout. 

11. Enhance reputation management initiatives by analyzing social media and blogs to gauge customer opinion and respond quickly to negative sentiments.

Customer Service 

12. Enable customers to easily access the latest information about products, including safety notices, manuals and warranties. 

13. Immediately answer questions about a new program, finding key details contained in emails, file shares and content repositories

Law Enforcement and Investigation 

14. Share critical information with other departments and agencies regardless of what format it’s in or where it’s stored. 

15. Discover the connections between relevant pieces of information hidden in any number of different sources

Legal 

16. Access facts in a case, pulling vital details out of every knowledge database and document or case management system available. 

17. Locate all intellectual property documents related to a new patent application and route them to the legal team.

Compliance 

18. Verify that information staff is sending overseas complies with international trade requirements. 

19. Search for relevant information and apply a hold to prevent modification or deletion, ensuring the content is locked down.

Contract Management 

20. Stay on top of expiring contracts and renewals, receiving automatic notifications when contracts are set to expire or milestones are due. 

21. Dramatically increase visibility to a large number of contracts stored in various locations, reporting on vital terms and figures. 

Accounting 

22. Gather all invoices contributing to excess spending on supplies and direct them to the right person for further review. 

23. Proactively identify all of the documents that could be from fraudulent sources or raise red flags during an audit.

Insurance 

24. Enable agents to analyze information from anywhere using mobile devices to uncover possible fraud before processing claims. 

25. Answer customer questions in one step, immediately locating policy, claim and payment details from multiple systems.

While every organization addresses information overload differently, these are best practices related to enterprise search technology that the most successful operations have in common:

1. Federate instead of consolidate.

As people create content at a dramatic rate across an organization, the costs, resources, and processes required to manage all of that information in one system have become prohibitive and unrealistic.

Organizations are finding that search technology can provide a single point of access to all types of information wherever it may exist, including legacy sources and thus delivering the benefits of consolidation without the obstacles.

2. Think organizationally, act individually.

With an enterprise search platform that is flexible and scalable enough to fit anywhere it is needed, successful organizations are empowering individuals to find and share information on their own. Distributing access to content eliminates knowledge silos, helps staff make better and more informed decisions, and saves time and resources.

3. Leave no document unturned.

At the top of the list of organization concerns is the accountability of all organizational content. Knowing what is floating around an organization, and being able to easily surface it, is crucial to mitigating risk and supporting compliance. Providing access to every ounce of relevant, unstructured information that exists outside of core business applications also is essential if you want processes to be fully informed.

4. Drive value from data.

Once they have unearthed their valuable information assets, smart organizations are immediately putting that data to use. Enterprise search can drive an operation's ability to analyze results, automate manual tasks and connect people to the right information at every point of need. It is not good enough to just find content you need to be able to do something with it.

5. Demand simplicity and usability.

When it comes to technology, especially enterprise search, users expect the experience to be simple, straightforward and pertinent to their roles. Top organizations understand those requirements because buy-in is critical. If a solution is too complex or the search results and functionality are not ideally suited to them, users will find workarounds. Then you will be back to square one.

Galaxy Consulting has over 15 years experience with enterprise search. Please contact us for a free consultation.

Intelligent Search Goes Beyond the Web

Search is a crucial component of the modern workplace. The ability to find information quickly and efficiently contributes not only to business success but also to employees satisfaction. 

It is frustrating to spend time looking for information when you could be completing a task.

Search has become ingrained as part of everyday life.

Pre-Internet Findability

Today, there’s no need to pull a volume of an encyclopedia off a shelf or even leave the room to find answers to questions. One can simply use phone to search for answers to questions. Google and Wikipedia have redefined what it means to search. But have they made search any more intelligent? They certainly satisfy the itch to correct people on event dates, geography, and historical characters. 

When it comes to the workplace, however, search encompasses a great deal more than fact checking, and intelligent search goes well beyond the web.

Search has gone mainstream. People use the word “search” when they want to locate a retail store or book a hotel. That simplistic notion of search does not carry over particularly well to finding information essential to doing your job.

Teasing Out the Meaning of the Search

Part of moving from a simple Google search to a more sophisticated model involves language. 

Standardizing content in one format—her example is high-definition PDFs—creates better visibility and fewer irrelevant search results. You may be able to avoid overly complex algorithmically based search engines by improving content processing, eliminating duplication, and using a single taxonomy.

Use better metadata and better data.

Almost anyone looking at search within the enterprise stresses findability. If you’re looking for the company’s holiday schedule, you don’t want the one from 3 years ago, you want the most recent one. 

Similarly, if you are building a web site for external use, you want potential customers to find what you are selling. You want to back up your sales efforts with excellent customer service. This is another opportunity for intelligent search, since customers increasingly prefer to help themselves without using an intermediary. They like self-service, but only if it answers their questions.

Semantics plays a role in customer service. Its analysis of the contextual meaning of words enhances the quality of answers. For example: customers might enter “How much will it cost me…” while your search engine understands phrases as “What is the price…” To be findable, your customer’s search query must translate to your words. Synonyms dictionary would help to resolve this issue.

Definition of intelligent search goes beyond findability. A search engine should know what you need and what your colleagues found valuable, and supply it to you when you need it. 

For Coveo search engine, the power and sophistication of machine learning technology is the driving force behind intelligent search. Intelligence springs from usage and analytics data, along with a multitude of other factors, the components of which are hosted and managed by companies such as Coveo.

Regardless of how you define intelligent search, it’s clear that enterprise search requirements go well beyond what Google or Wikipedia can provide. Different approaches to intelligent search provide much to think about when implementing, redesigning, and rethinking enterprise search. Intelligent search goes well beyond what searching the web looks like.

Improving Search and Decision-Making with Semantics

We’ve all heard about how Google’s proverbially simple search form has led professionals to expect similar simplicity from search solutions provided by corporate IT. Except this model doesn’t really work, and it’s costing millions of dollars every year in time wasted when professionals don’t find, and have to re-create information.

The reason it doesn’t work is that while every organization has a specific worldview, search engines are essentially blind. Worldview is the inventory of business objects that an organization cares about (products, geographies, customers, processes, etc.) and their relationships, that are typically captured in a taxonomy or ontology. While professionals implicitly want to search for information according to their worldview, search engines don’t offer them a practical way to do so.

Semantics Provides Meaning

The missing piece in this puzzle is a “meaning engine” that would understand unstructured content through the lens of your organization’s worldview. It exists: it’s called a semantic enrichment platform.

A semantic enrichment platform ingests your organization’s taxonomy or ontology and applies it to your content at scale. Leveraging natural language processing, it understands your content the same way humans do. It recognizes topics that are relevant to your business, entities of interest, their attributes and relationships, and converts them into structured data, that can be used standalone, or as metadata describing your content deeply and consistently. In energy, for example, entities of interest might include commodities, trading companies, and the countries where they do business.

Better Metadata Accelerates Search

When used as metadata, this data acts as an eye-opener for search engines that can finally see your content through your own worldview. This redefines the search experience by offering end-users new tools to locate what they are looking for.

Faceted Navigation enables end-users to search by business entity or topic (for example by company name, commodity type or region), helping to find the most relevant content in just a few clicks.

Links to relevant information provide convenient access to structured information about entities of interest so users don’t have to collate it themselves. For example, each company name could be linked to data about its activities. Topic pages concentrate all information about a specific topic in one convenient access point so users don’t need to sift through all other materials to access it. A topic page on electricity would, for example, filter out information related to other energy sources.

Content recommendation uses metadata to surface other documents with similar topics, promoting discovery of relevant information. A document on a merger in the gas sector might point to reports of other, similar operations.

Such mechanisms significantly accelerate and simplify search tasks, offering not only time and cost savings, but also more informed decision-making.

Better Data Improves Decision-Making

But semantically-extracted information can be used for its informative, rather than descriptive, value. Not as metadata, but as standalone data. This opens the door to applications that address the above blindness at a deeper level, providing higher-level and faster insight into the subject matter at hand.

One of semantics’ capabilities is to recognize not only entities, but also their relationships (often expressed as triples). One such relationship might for example indicate that company A is a “supplier of” company B. Information value from these relationships may come into play under a variety of scenarios.

Knowledge Bases (or Graphs) integrate such structured information at scale so they can then be queried. One might contain, for a given commodity, links to all suppliers.

Complex Reasoning can be performed on these knowledge bases, enabling business applications to provide higher degrees of automation in decision-making tasks, for example, automatically balancing supply by identifying alternative suppliers when one announces production issues.

Analytics and Visualizations provide dashboards that sit on top of the data and reveal its meaning on a more holistic level. For example, a network graph could plot all company relationships in natural gas, indicating which companies might be exposed to increasing prices in a given region.

Lastly, semantics can also be used to deliver Question Answering Systems that offer users a way to get answers to questions formulated in natural language (“Which electricity providers have the most diversified supply chain?”) instead of engaging in search.

Semantics Provides Faster Insights and Better Decisions

As can be seen from the examples above, semantics is the “meaning engine” that ensures that users can overcome search’s blindness and access information through the specific worldview relevant to their work. But this engine brings meaning to more than your search engine: it is your information management as a whole that benefits, bringing the promise of smarter applications that efficiently handle more of the groundwork, accelerate time-to-insight and support better decisions.

Self-learning

Intelligent search is no longer a nice-to-have feature in organizational information systems; it is a critical part of how businesses are transforming the way they work. Intelligent search goes beyond findability and information access. Like a trusted advisor, intelligent search knows what documents you need for your tasks and which articles your colleagues found most valuable and would be useful to you too, and simply gives everyone the information they need, when they need it. And the power and sophistication of machine-learning technology is the driving force behind intelligent search.

What Is Machine Learning?

Machine learning learns from and makes predictions on data. Applied to search, every time a user performs an action on your web site or support portal, he or she provides data about what is useful. Did they submit a support ticket? That means the articles they just read did not help. Do most people spend only one minute with a document that would normally take 10 minutes to read? That’s a sign that the content isn’t useful, or perhaps it’s too difficult to understand. With machine learning, all of that information and more can be used to make data-driven predictions and decisions without manual intervention.

How Will Machine Learning Make Search Intelligent?

When someone submits a search query or clicks on the third search result, they are implicitly telling you what is most relevant. As your online community members download content, visit various web pages, watch videos, start an online chat with your support agents or submit support tickets, their behavior provides information on the relevance of the content they come across. This behavioral data as well as search behavior which signals intent are captured by search usage analytics.

Intelligent self-learning search engines powered by machine learning can leverage such usage analytics data to continuously self-learn. This improves search relevance and hence, the self-service experience on your community in many ways. For example, automatic fine-tuning and ranking of search results based on machine-generated predictions about what is most useful improves the experience of all community members.

Without machine learning and analytics data, administrators need to fine-tune search rankings manually: create boosting rules, add synonyms, promote documents, etc. Because relevance is an ever-evolving process and the document that was the most relevant last week may no longer be relevant today, it is almost impossible for administrators, especially those at large organizations or those with multiple product lines, to keep pace with the rate of change.

With machine learning, highly manual and complex enterprise search can be transformed into intelligent, self-learning and self-tuning search.

Why Now?

Machine learning has been around for a long time. It used to be very complex to deploy and manage. Collecting usage data, managing databases, provisioning servers, developing and maintaining machine learning algorithms and using machine-learning predictions in the search system were typically very complex. This would require data scientists, database experts and developers. Only the biggest organizations could afford that. But the fast adoption of cloud solutions has made the use of machine learning much easier, cheaper and more attainable. In particular, the recent trend towards cloud-based enterprise search is a game changer.

What Is the Impact of Cloud-Based, Self-Learning Search?

With cloud-based, self-learning search, all the required components are hosted and managed by the vendor, such as Coveo. Because of its scalability, it has the potential to change the customer service industry the same way machine learning has impacted e-commerce and social networks. 

In the past, the high cost of using and managing machine-learning systems meant that machine learning was rarely used for traditional enterprise search or self-service support sites. The cloud makes that affordable to all customers and to all departments, especially when deploying self-learning search on self-service support sites and on communities, because of its ability to scale and handle large volumes of data.

Specialized Strategies for Enterprise Search

Enterprise search is the practice of making content from multiple enterprise sources such as databases and intranets, searchable to a defined audience. "Enterprise search" is also used to describe the software of search information within an enterprise.

Enterprise search systems index data and documents from a variety of sources such as file systems, intranets, content management systems, e-mail, and databases. Many enterprise search systems integrate structured and unstructured data in their collections. Enterprise search systems also use access controls to enforce a security policy on their users.

Enterprise search as a standalone application for locating documents throughout the enterprise is still going strong, but many search engines are now embedded in applications that people use as their primary work environment. Search solutions are also used on more complex tasks such as locating relevant information found in either databases or diverse document repositories.

New search tools are emerging for searching geospatial data and processing it with other types of information to provide an enriched and informative blend.

SAVO

One of the new enterprise search applications is SAVO.

SAVO focuses on driving greater sales productivity, and one way of doing this is providing the content that is needed by sales reps in context. Content can be stored in such a way where users can access it either locally or remotely. The SAVO platform provides search capabilities in a content repository, which contains approved information designed to support salespeople as they do their jobs. The search function is configured with a profile for each salesperson to push out the information that will be most useful to each individual.

By using profiles, overall selling methodology and model can be reinforced. The information pushed out might include a list of documents for a client meeting, a video clip, information by product line or a summary of facts about a competitor. To support the sales reps’ conversations, SAVO provides context-relevant information.

Although content for different phases of the sales cycle is predefined, if unexpected questions arise, sales reps need to have the right information at their fingertips.

By setting up rules for tagging and reviewing information as it is added to the repository, SAVO is able to keep the right information in front of the sales rep. All the information they need is in the backend. The strategy is not to present the rep with a blank search box. Instead, at the frontend, the rep is asked a series of questions such as what product they are dealing with, what type of document they want, whether it’s a customer-facing presentation or a brochure. In two or three clicks, they have reduced a potential list of 60 documents to about five.

Also searchable on the SAVO platform is previously tacit knowledge that has been captured in forums or in comments. Referred to as “tribal knowledge” by SAVO, the content is generated by employees who post questions and answers on a subject matter expert moderated forum within SAVO. That content can be sought on a proactive basis by the sales reps when they have a specific question. Between the profiled information and the responses to ad hoc queries, the sales reps are able to respond with accurate and timely information.

The search function is enabled on mobile devices as well as the desktop. Because sales is both a global and regional activity, SAVO supports multiple languages, including ideographic languages such as Japanese and Chinese, and the search function is available for all the foreign language versions of the product.

The search capability provided by SAVO is useful not only for salespeople in the field but also in onboarding. It helps new employees who are learning the ropes, because they can access instructional material, reference material and conversations posted by more experienced sales reps.

DtSearch

DtSearch is effective because it is a mature product with a very stable API.

Search applications with complex data models pose special challenges. One such example was a search application Contegra built for Carrier Corporation. The number of parts is large, and many parts have revisions to them that have been built over the years. Serial numbers, model numbers and other identifiers must be incorporated. Also, the system had to be able to identify what product the customer originally received because that would affect the replacement part needed. In addition to searching the database, the application must be able to locate technical documentation, brochures and other files in a variety of formats.

DtSearch is able to search across SQL databases and PDF documents, and then search parts referenced within documents. It sounds like a simple task to recognize parts numbers, but when you have 50,000 of them with complex relationships and want to do the search quickly, using the right search tool is very important.

It is able to understand the search needs of different users. Engineers see the world as a set of different systems, such as engines and drives systems, which have specific code numbers. There are hierarchical taxonomies, model numbers and serial numbers. Retrieving the exact information for each installation is critical when considering the action being taken is a certain type of repair or service. Each user group could have a different perspective on the data that requires it to be searched and viewed differently.

Particularly in specialized applications, the upfront work is critical. You would want to be sure that when the users do a keyword search, the information is presented in a meaningful context, so they can narrow it down to the exact part and the nature of the document, whether it is a technical publication, leaflet or catalog. A good content model enables both the field service staff and external customers to access information in a self-service mode, which cuts costs for the company.

Geospatial Search

Geospatial information has become increasingly important for many different applications and analyses ranging from marketing to agriculture. Yet the management of geospatial information has lagged that of any other kind of information. Although maps and imagery can be stored in a repository like any other digital files and searched according to indexed metadata, the ability to perform more complex searches on the data and process it once retrieved has been limited.

Voyager Search is a search solution designed specifically to manage spatial information. It combines modern search technologies with a unique understanding for geospatial data.

It includes an indexing solution that can extract information from nearly any type of geospatial data regardless of format. It later began supporting other non-spatial documents such as PDF and various Microsoft Office formats. It also offers a solution that enriches data through linking documents to a map.

Users are able to define a geographical area on a map and then search for relevant information about it. The user can find all the river data or stream flow data in that area, for example, or reports by dragging a box on a map. This ability is not available in other search engines.

Voyager Search can manage very large quantities of spatial data. Making data available and accessible while keeping it secure is a big challenge. A geospatially enabled search solution is critical not only to providing online access to geospatial intelligence, but also to broadening the analytical expertise of the organization. Voyager accomplishes that by providing the tools to index a wide variety of content and to add a layer of geospatial intelligence to the index.

The process for narrowing results from a search of geospatial data differs from that of other types of data. Search results from a demographics database, for example, might be narrowed by selecting only one income category. In a spatial search, “refine your search” might mean “view only content in a specific geography” in order to look at one aspect of the results.

Voyager Search can also create new files derived from geospatial and image data. For example, a petroleum engineer could look for seismic shockwave data from a certain time interval for use in the analysis. Emergency responders might look for recent areal photographs taken in their area and run them through an imagery analysis process to look for hot spots that would indicate the most recent fire perimeter. Voyager Search offers that capability, known as “geoprocessing,” which includes a variety of ways to manipulate geospatial data at the desktop.

These are just few new search applications currently on the market. Galaxy Consulting has over 17 years experience in enterprise search and enterprise search applications.

Search in the Land of Information Silos

Information access and retrieval within most organizations is a work in progress. There might be a general search system for marketing information, and probably one or more database search systems.

The larger the organization, the greater the number of information retrieval systems. Each laptop and mobile device has a search system. Mobile phone apps sport their own search systems. The lawyers in an organization may have different search systems for specific types of legal matters. The enterprise resource planning (ERP) users have a search system. When it comes to enterprise search, there are many silos.

A “silo” is a content collection available to certain users. In the face of the reality of silos, it might be impractical idea of providing access to “all” information. “All” may not mean all or even some available information. Big data is easy to talk about but difficult to make accessible. The same challenge exists for images, audio recordings, and engineering drawings with details hidden into the proprietary system’s database.

Search which is variously called universal, unified or federated search is a solution to the challenge of information silos. The term meta-search is often used to describe an integrating function that passes the user’s query across discrete content indexes and returns a single results list to the user. Endeca, Inxight Software, Northern Light, Sagemaker and Vivisimo are search applications that can be used for universal, unified or federated search in an organization.

The initial query might not unlock the information stored in the system’s index. The facets, topics and suggests make it easy for the user to click through the links without having to craft additional queries.

Behind the curtains, federated search results requires some maintenance. A user does not want to know the file format in which the information he or she needs is stored. The user wants answers. Early federating systems like WAIS relied on standards for content representation. Today, however, there are many “standards,” and content processing systems must be able to process content in the hundreds of formats found in organizations.

It is important to deliver a system that makes an organization’s disparate types of digital content available.

There are barriers to unified, federated or integrated search.

Some digital content cannot be included in a general purpose search system for security, business or legal reasons. Technical content such as chemical structure information at a pharmaceutical company requires special purpose systems. The same need applies to product manufacturing data, legal information and engineering drawings.

Most search applications exclude video streams from the index. If video is indexed, the system processes the text included in the digital file or indexing provided by the video owner.

The cost of creating connectors to connect with certain content types could be too high, or license fees could be required to gain access to the file formats.

The computational burden required to process certain types of content might exceed the organization’s ability to fund the content processing. Big data, for example, requires a computing capability able to handle the Twitter stream, RSS feeds and telemetry data from tracking devices. Cost could be prohibitive for processing all content types.

The most important challenge is the need for confidentiality. The legal department does not want unauthorized access to information related to a legal matter out of its control.

Some government contracts required that for certain types of government work, the information related to that project must be separated. Common sense dictates that plans for a new product and its pricing remain protected. If someone needs access to that information, a different search system may be used to ensure confidentiality.

Even in the absence of business or legal requirements, some professionals do not want to share content. That may be a management problem. When a manager locks up information in a no-access silo, a software script will skip the flagged server.

To summarize, silos of information present a challenge to process and effectively use in organizations. In the enterprise, integration should take place within silos of content.

Galaxy Consulting has 17 years experience in integrating information silos using universal, unified or federated search. We have experience with search applications. Contact us for no obligation free consultation!

Search Applications - Concept Searching

Concept Searching Limited is a software company which specializes in information retrieval software. It has products for Enterprise search, Taxonomy Management and Statistical classification.

Concept Searching Technology Platform

The Concept Searching Technology Platform is based on our Smart Content Framework™ for information governance, and incorporates best practices for developing an enterprise framework to mitigate risk, automate processes, manage information, protect privacy, and address compliance issues. Underlying the framework is the technology to:

• Automatically generate semantic metadata using Compound Term Processing.
• Auto-classify content from diverse repositories.
• Easily develop, deploy, and manage taxonomies.

The framework is being used to enable intelligent metadata enabled solutions to improve search, records management, enterprise metadata management, text analytics, migration, enterprise social networking, and data security.

Features

• Compound terms are extracted when content is indexed from internal or external content sources, enabling the delivery of greater precision of relevant content at the top of search results.
• Relevance ranking displays extracts from the documents based on the query.
• Search refinement delivers to the end user highly correlated concepts that may be used to refine the search.
• Taxonomy browse capabilities are standard.
• Documents can be classified into one or more taxonomy nodes, enhancing the precision of documents returned.
• In addition to static summaries, Dynamic Summarization, a modified weighting system, can be applied that will identify in real-time short extracts that are most relevant to the user’s query.
• Related Topics will return results based on the conceptual meaning of the search terms used, using the ability to generate compound terms in a search. For example, ‘triple’ is a single word term but ‘triple heart bypass’ is a compound term that provides a more granular meaning.
• Based on previous queries, or on extracts retrieved, end users can use the text to perform additional searches to retrieve more granular results.
• The product is based on an open architecture with all API’s based on XML and Web Services. Transparent access to system internals including the statistical profile of terms is standard.
• Highly scalable.
• High performance specifically with classification occurring in real time.
• Easily customized to achieve your organizations’ objectives.

Base Components in the Concept Searching Technology Framework

Conceptual Search Platform

conceptSearch, is Concept Searching’s enterprise search product and a key component in the Concept Searching Technology Platform. It is a unique, language independent technology and is the first content retrieval solution to integrate relevance ranking based on the Bayesian Inference Probabilistic Model and concept identification based on Shannon’s Information Theory.

Unlike other enterprise search engines that require significant customization with marginal results, conceptSearch is delivered with an out-of-the-box application that demonstrates a simple search interface and indexing facilities for internal content, web sites, file systems, and XML documents. Application developers experience a minimal learning curve and the organization can look forward to a rapid return on investment.

Because of the innovative technology, conceptSearch delivers both high precision and high recall. Precision and recall are the two key performance measures for information retrieval. Precision is the retrieval of only those items that are relevant to the query. Recall is the retrieval of all items that are relevant to the query. Yet most information retrieval technologies are less than 22% accurate for both precision and recall. The ideal goal is to have these features balanced. Compound term processing has the ability to increase precision with no loss of recall.

conceptSearch is particularly important for organizations that need sophisticated search and retrieval solutions. By weighting multi-word phrases, instead of single words, or words in proximity, the retrieval experience is more accurate and relevant. The ability for the search engine to identify concepts enables organizations to improve the search experience for a variety of business requirements.

Search Engine Integration

This functionality is provided via the Concept Searching Technology platform to integrate with any search engine. The Concept Searching Technology platform can perform as on the fly classification with search engines calling the classify API. Search engine support includes SharePoint, the former FAST products, Office 365 Search, Solr, Google Search Appliance, Autonomy, and IBM Vivisimo. If the FAST Pipeline Stage is required, this is sold as a separate product.

conceptClassifier

conceptClassifier is a leading-edge rules based categorization module providing control of rules-based descriptors unique to an organization. conceptClassifier delivers a categorization descriptor table, which is easy to implement and maintain, through which all rules and terms can be defined and managed. This approach eliminates the error-prone results of ‘training’ algorithms typically found in other text retrieval solutions and enables human intervention to effectively tune classification results.

Functionality is provided via the Concept Searching Technology platform, to classify documents based upon concepts and multi-word terms that form a concept. Automatic and/or manual classification is included. Knowledge workers with the appropriate security rights can also classify content in real time. Content can be classified from diverse repositories including SharePoint, Office 365, file shares, Exchange public folders, and websites. All content can be classified on the fly and classified to one or more taxonomies.

conceptTaxonomyManager

This is an advanced enterprise class, easy-to-use taxonomy development and management tool, still unique in the industry. Developed on the premise that a taxonomy solution should be used by business professionals, and not the IT team or librarians, the end result is a highly interactive and powerful tool that has been proven to reduce taxonomy development by up to 80% (client source data).

conceptTaxonomyManager is a simple to use, has an intuitive user interface designed for Subject Matter Experts, and does not require IT or Information Scientist expertise to build, maintain and validate taxonomies for the enterprise. conceptTaxonomyManager has the capability to automatically group unstructured content together based on an understanding of the concepts and ideas that share mutual attributes while separating dissimilar concepts.

This approach is instrumental in delivering relevant information via the taxonomy structure as well as using the semantic metadata in enterprise search to reduce time spent finding information, increase relevancy and accuracy of the search results, and enable the re-use and re-purposing of content. Using one or more taxonomies, unstructured content can be leveraged to improve any application that uses metadata. This flexibility extends to records management, information security, migration, text analytics, and collaboration.

Intelligent Migration

Using the Concept Searching Technology platform an intelligent approach to migration can be achieved. As content is migrated it is analyzed for organizationally defined descriptors and vocabularies, which will automatically classify the content to taxonomies, or in the SharePoint environment, the SharePoint Term Store, and automatically apply organizationally defined workflows to process the content to the appropriate repository for review and disposition.

conceptSQL

This product provides the ability to define a document structure based on information held in a Microsoft SQL Server. A document can include any number of text and metadata fields and can span multiple tables if required. conceptSQL supports SQL 2005, 2008, and 2012. A powerful but easy to use configuration tool is supplied eliminating the need for any programming. Templates are provided for out of the box support for Documentum, Hummingbird, and Worksite/Interwoven DMS.

SharePoint Feature Set

The SharePoint Feature Set includes the following components: farm solution with feature sets, Term Store integration, taxonomy tree control for editing, refinement panel integration, event handlers for notification of changes, management of classification status column, web service advanced functionality (implement system update or preserve GUIDS), automated site column creation.

Intelligent Records Management

The ability to intelligently identify, tag, and route documents of record to either a staging library and/or a records management solution is a key component in driving and managing an effective information governance strategy. Taxonomy management, automatic declaration of documents of record, auto-classification, and semantic metadata generation are provided via the Concept Searching Technology platform and conceptTaxonomyWorkflow.

Data Privacy

Fully customizable to identify unique or industry standard descriptors, content is automatically meta-tagged and classified to the appropriate node(s) in the taxonomy based upon the presence of the descriptors, phrases, or keywords from within the content. Once tagged and classified the content can be managed in accordance with regulatory or government guidelines.

The identification of potential information security exposures includes the proactive identification and protection of unknown privacy exposures before they occur, as well as real-time monitoring of organizationally defined vocabulary and descriptors in content as it is created or ingested. Taxonomy, classification, and metadata generation are provided via the Concept Searching Technology platform and conceptTaxonomyWorkflow.

eDiscovery and Litigation Support

Taxonomy, classification, and metadata generation are provided via the Concept Searching Technology platform. This is highly useful when relevance, identification of related concepts, vocabulary normalization are required to reduce time and improve quality of search results.

Text Analytics

Taxonomy, classification, and metadata generation are provided via the Concept Searching Technology platform. A third party business intelligence or reporting tool is required to view the data in the desired format. This is useful to cleanse the data sources before using text analytics to remove content noise, irrelevant content, and identify any unknown privacy exposures or records that were never processed.

Social Networking

Taxonomy, classification, and metadata generation are provided via the Concept Searching Technology platform. Integration with social networking tools can be accomplished if the tools are available in .NET or via SharePoint functionality. This is useful to provide structure to social networking applications and provide significantly more granularity in relevant information being retrieved.

Business Process Workflow

conceptTaxonomyWorkflow serves as a strategic tool managing migration activities and content type application across multiple SharePoint and non-SharePoint farms and is platform agnostic. This add-on component delivers value specifically in migration, data privacy, and records management, or in any application or business process that requires workflow capabilities.

conceptTaxonomyWorkflow is required to apply action on a document, optionally automatically apply a content type and route to the appropriate repository for disposition.

Latest Applications in Enterprise Search

In my previous post, I described the future of enterprise search. In this post, I will describe few new search applications that could be interesting.

Concept Searching

Founded in 2002, Concept Searching provides software products that deliver automatic semantic metadata generation, auto-classification, and powerful taxonomy management tools. Concept Searching is the only platform independent statistical metadata generation and classification software company in the world that uses concept extraction and compound term processing to significantly improve access to unstructured information. The Concept Searching Microsoft suite of technologies runs in all versions of SharePoint, Office 365, and OneDrive for Business.

The technologies are being used to improve search outcomes, deploy an enterprise metadata repository, enable effective records management, identify and secure sensitive information, improve governance and compliance, social tagging, collaboration, text analytics, facilitate eDiscovery, and drive intelligent migration.

Concept Searching, developer of the Smart Content Framework™, provides organizations with a method to mitigate risk, automate processes, manage information, protect privacy, and address compliance issues. This infrastructure framework utilizes a set of technologies that encompasses the entire portfolio of unstructured information assets, resulting in increased organizational performance and agility.

Lexalytics, Inc.

Lexalytics provides enterprise and hosted text analytics software to transform unstructured text into structured data. The software extracts entities (people, places, companies, products, etc.), sentiment, quotes, opinions, and themes (generally noun phrases) from text. Text is considered unstructured data which comprises somewhere between 31% and 85% of what is stored in any given enterprise.

Lexalytics is an OEM vendor of text analytics and sentiment analysis technology for social media monitoring, brand management, and voice-of-customer industries. The software uses natural language processing technology to extract the above-mentioned items from social media and forums; the voice of the customer in surveys, emails, and call-center feedback, traditional media, pharmaceutical research and development, internal enterprise documents, and others.

Lexalytics, provides a text mining engine that is used by a number of search partners like Coveo, Playence, and Oracle to add additional metadata to their search. This is additional intelligence around "just what do those words actually mean?" In other words, this engine is boosting the value of search by providing more information into the index. This enables other applications, and helps search be "smarter".

MaxxCAT

MaxxCAT provides enterprise search solutions for corporate intranets, web sites, databases, file systems and applications, and other environments that require rapid document retrieval from multiple data sources. The flagship products offered by MaxxCAT are the SB-250 series and the EX-5000 series network search appliances. Also available are series of cloud-enables storage appliances.

Basis Technology

Founded in 1995, this software company specializes in applying artificial intelligence techniques to understanding documents written in different languages. Their software enhances parsing tools by classifying the role of words and provides metadata on the role of words to other algorithms. Software from Basis Technology will, for instance, identify the language of an incoming stream of characters and then identify the parts of each sentence like the subject or the direct object.

The company is best known for its Rosette Linguistics Platform which uses Natural Language Processing techniques to improve information retrieval, text mining, search engines and other applications. The tool is used to create normalized forms of text by major search engines, and, translators. Basis Technology software is also used by forensic analysts to search through files for words, tokens, phrases or numbers that may be important to investigators.

dtSearch

Founded in 1991, this company specializes in text retrieval software. Its current range of software includes products for enterprise desktop search, Intranet/Internet spidering and search, and search engines for developers (SDK) to integrate into other software applications

LTU technologies

Founded in 1999, this company is in the field of image recognition for commercial and government customers. The company provides technologies for image matching, similarity and color search for integration into applications for mobile, media intelligence and advertisement tracking, ecommerce and stock photography, brand and copyright protection, law enforcement and more

Sematext Group, Inc.

This company's product SSA - Site Search Analytics - continuously monitors, measures, and improves the search experience. It identifies top queries, problematic zero-hit queries, common misspellings, etc. It measures and compares search relevance and improves conversion rates. It is available It is available on-premises and in the cloud.

Exorbyte

This is a privately held software company which was founded in 2000 in Konstanz, Germany, with an additional office in the United Kingdom (Bristol). The company develops intelligent software for search and analysis of structured and semi-structured data.

Their product MatchMaker is the leading error-tolerant search & match platform for huge master data volumes. The multiple award-winning software technology thinks, searches and finds like a human – but dramatically faster, in much more complex configurations and with no serious data restriction using keys or similar methods. It is available on-premises and in the cloud.

Federal authorities, insurance agencies, ICT firms and more use this software to identity a resolution in diverse, data-intensive business processes such as input management, enterprise search and data quality. It has easy customization and integration.

Inbenta

Founded in 2005,this company provides enterprise semantic search technology based on artificial intelligence and natural language processing. It offers intuitive search solutions and intelligent content support for website and corporate Intranets.

Content Analyst Company

This is a privately held software company which develops concept-aware text analytics software called CAAT, which is licensed to software product companies for use in eDiscovery. In 2013, five CAAT-powered products were named in the Gartner eDiscovery Magic Quadrant Report, and the analyst firm 451 Group referred to CAAT as The Hottest Product in eDiscovery.

Content Analyst's CAAT analytics software is a machine learning system based on latent semantic indexing technology. CAAT provides several text analytics capabilities using both supervised learning and unsupervised learning methods including concept search, categorization, conceptual clustering, email conversation threading, language identification, near-duplicate identification, auto summarization and difference highlighting.

SearchYourCloud

With SearchYourCloud and its patented, federated search technology, a single search request in Outlook simultaneously and transparently searches your email, desktop and all of your cloud storage sources and delivers highly targeted results. You get exactly the information you need with just one query.

Docurated

Docurated aggregates all your documents in one place, turning them into a searchable and customizable database. Docurated will now provide Dropbox integration as well. It accelerates sales in companies looking for fast growth by making the best marketing content readily available to Sales around the world. Docurated works with your existing content stores and uses machine learning to enable your team to find and re-use the most effective content with no manual tagging or uploading.

This is the next generation visual knowledge management platform which solves the information retrieval problem for leading companies like Clorox, Omnicom, Netflix, Weather Channel, and many others. Docurated enables sales, marketing, and technology teams to surface and use the exact chart or slide they need, no matter where it is stored, without slogging through folders and files. Docurated seamlessly integrates with existing folder-based repositories.

Lucene

Apache Lucene is a free open source information retrieval software. It is supported by the Apache Software Foundation and is released under the Apache Software License. While suitable for any application which requires full text indexing and searching capability, Lucene has been widely recognizedfor its utility in the implementation of Internet search engines and local, single-site searching.

At the core of Lucene's logical architecture is the idea of a document containing fields of text. This flexibility allows Lucene's API to be independent of the file format. Text from PDFs, HTML, Microsoft Word, and OpenDocument documents, as well as many others (except images), can all be indexed as long as their textual information can be extracted.

These are just few search applications that are currently on the market. There are many others. Choosing the right application is based on your organization's requirements.

Future of Enterprise Search

Enterprise search is a developing industry. In this post, I will describe the latest developments in enterprise search.

Effective enterprise search represents one of the most challenging areas in business today. The whole area of search has been revolutionized by Google. Employees now expect to be able to locate relevant data as easily as they navigate the web through Google. When this ease of search is not replicated in organizations' systems, it can be quite frustrating. As we create more content than ever before, the importance of effective search across the enterprise continues to grow.

Until recently, much of the enterprise search technology remained unchanged. The general purpose enterprise search offerings were fairly similar in technology and scope. There are now many software companies who direct their efforts towards enterprise search. The future will bring shorter innovation cycles, continuous user experience improvements, deeper integration with first- and third-party applications and more ETL-like (extract, transform and load) functionality to handle poor quality content.

In the second half of the 2000’s, the enterprise search companies were absorbed by the large software companies:

• Microsoft acquired FAST Search in 2008
• Adobe acquired Mercado in 2009
• Dassault Systèms acquired Exalead in 2010
• Hewlett Packard acquired Autonomy in 2011
• Oracle acquired Endeca in 2011
• IBM acquired Vivisimo in 2012

User experience is a broad topic in itself, with active trends including:

• Richer information about the user to determine context, such as their business context, social context, mobile device sensors, location, speech recognition, preferences and historical usage.
• Advances in visualization such as HTML 5.
• Natural language processing as in the trends seen with Wolfram Alpha and smart phone digital assistants, such as Apple’s Siri, Microsoft’s Cortana and Google Now.
• Richer results that look less like a page of links and more like answers to questions.
• Elements of knowledge management that add meaning to queries and results.
• Enterprise search products will become increasingly and more deeply integrated with existing platforms, allowing more types of content to be searchable and in more meaningful ways. It will also become less of a dark art and more of a platform for discovery and analysis.

The future of enterprise search seems destined to continue with simple keyword and Boolean searching, augmented by faceted navigation based on metadata. Virtually every e-commerce web site today offers guided navigation based on metadata.

This ubiquitous model now appears in most of the leading enterprise search products and users immediately understand how a simple text query can quickly be focused to a specific domain by clicking on a metadata filter. This updated search model is increasing demand for auto-classification products which can generate descriptive metadata automatically based on an analysis of the document’s unstructured content.

Open source software has made significant improvements, displacing many of the traditional search vendors. Lucene and its supporting companies like LucidWorks provide solid search functionality at a hard-to-beat price. Where vendors are seeing success is in four main areas:

• Providing functionality beyond typical "search" – extending to facets, true knowledge management, multimedia search, and other functionality.
• Focusing on vertical-specific applications like fraud and supply-chain management.
• Working with larger, more conservative enterprises.
• Providing a SaaS, one-stop-shop for zero (or low) touch functionality.

A few major factors are going to drive the industry going forward:

• Open source will continue to get better and drive out inefficiency in the market .
• More, better information about the searcher: location awareness, profile sharing, time dependence, deeper understanding of the context and content of the search. With this information, you can provide better, more relevant results. 
• Lower tolerance for hassle: people expect search to "just work" – not understanding that it can be just as complicated as any other major IT initiative. By having low-touch solutions, SaaS providers will make major progress in the small/medium business world.
• Search all the things!: Integrated understanding of objects, video, speech, as well as traditional semantic sources like text will combine together better into a whole that allows for information retrieval no matter what the format.

Another area for future development is machine to machine consumption of information and sharing. Search providers are increasingly applying advanced analytics of text and other media so their users’ desires are more deeply satisfied through relevant search results. Search will be increasingly entity-centric and collaborative.

Future of search will include more semantic understanding of both content and queries. For example Exorbyte is focused on searching in structured master data – people, products and places, and its ability to query this data without use of restrictive match-keys for both lexicographical and semantic similarity is globally unique.

The future of search goes through natural language processing while on the other hand it will entail the capability of providing advanced information analysis during indexation time.

The facility to search within the document itself is becoming vital. The Docurated platform caters for instant access to the most relevant page or slide without even having to open the document.

Effective enterprise search can eradicate inefficiency. Enterprise search will become instant and intuitive, paving the way for increased productivity across the enterprise.

In my next post, I will highlight few search applications that could be worth looking into...

Search Applications - Vivisimo

Vivisimo was a privately held technology company that worked on the development of computer search engines. The company product Velocity provides federated search and document clustering. Vivisimo's public web search engine Clusty was a metasearch engine with document clustering; it was sold to Yippy, Inc. in 2010.

The company was acquired by IBM in 2012 and Vivisimo Velocity Platform is now IBM InfoSphere Data Explorer. It stays true to its heritage of providing federated navigation, discovery and search over a broad range of enterprise content. It covers broad range of data sources and types, both inside and outside an organization.

In addition to the core indexing, discovery, navigation and search engine the software includes a framework for developing information-rich applications that deliver a comprehensive, contextually-relevant view of any topic for business users, data scientists, and a variety of targeted business functions.

InfoSphere Data Explorer solutions improve return on all types of information, including structured data in databases and data warehouses, unstructured content such as documents and web pages, and semi-structured information such as XML.

InfoSphere Data Explorer provides analytics on text and metadata that can be accessed through its search capabilities. Its focus on scalable but secure search is part of why it became one of the leaders in enterprise search. The software’s security features are critical, as organizations do not want to make it faster for unauthorized users to access information.

Also key is the platform’s flexibility at integrating sources across the enterprise. It also supports mobile technologies such as smart phones to make it simpler to get to and access information from any platform.

Features and benefits

1. Secure, federated discovery, navigation and search over a broad range of applications, data sources and data formats.

• Provides access to data stored a wide variety of applications and data sources, both inside and outside the enterprise, including: content management, customer relationship management, supply chain management, email, relational database management systems, web pages, networked file systems, data warehouses, Hadoop-based data stores, columnar databases, cloud and external web services.
• Includes federated access to non-indexed systems such as premium information services, supplier or partner portals and legacy applications through the InfoSphere Data Explorer Query Routing feature.
• Relevance model accommodates diverse document sizes and formats while delivering more consistent search and navigation results. Relevance parameters can be tuned by the system administrator.
• Security framework provides user authentication and observes and enforces the access permissions of each item at the document, section, row and field level to ensure that users can only view information they are authorized to view in the source systems.
• Provides rich analytics and natural language processing capabilities such as clustering, categorization, entity and metadata extraction, faceted navigation, conceptual search, name matching and document de-duplication.

2. Rapid development and deployment framework to enable creation of information-rich applications that deliver a comprehensive view of any topic.

• InfoSphere Data Explorer Application Builder enables rapid deployment of information-centric applications that combine information and analytics from multiple sources for a comprehensive, contextually-relevant view of any topic, such as a customer, product or physical asset.
• Widget-based framework enables users to select the information sources and create a personalized view of information needed to perform their jobs.
• Entity pages enable presentation of information and analytics about people, customers, products and any other topic or entity from multiple sources in a single view.
• Activity Feed enables users to "follow" any topics such as a person, company or subject and receive the most current information, as well as post comments and view comments posted by other users.
• Comprehensive set of Application Programming Interfaces (APIs) enables programmatic access to key capabilities as well as rapid application development and deployment options.

3.Distributed, highly scalable architecture to support large-scale deployments and big data projects.

• Compact, position-based index structure includes features such as rapid refresh, real-time searching and field-level updates.
• Updates can be written to indices without taking them offline or re-writing the entire index, and are instantly available for searching.
• Provides highly elastic, fault-tolerant, vertical and horizontal scalability, master-master replication and “shared nothing“ deployment.

4. Flexible data fusion capabilities to enable presentation of information from multiple sources.

• Information from multiple sources can be combined into “virtual documents“ which contain information from multiple sources.
• Large documents can be automatically divided into separate objects or sub-documents that remain related to a master document for easier navigation and comprehension by users.
• Enables creation of dynamic "entity pages" that allow users to browse a comprehensive, 360-degree view of a customer, product or other item.

5. Collaboration features to support information-sharing and improved re-use of information throughout the organization.

• Users can tag, rate and comment on information.
• Tags, comments and ratings can be used in searching, navigation and relevance ranking to help users find the most relevant and important information.
• Users can create virtual folders to organize content for future use and optionally share folders with other users.
• Navigation and search results can return pointers to people to enable location of expertise within an organization and encourage collaboration.
• Shared Spaces allow users to collaborate about items and topics that appear in their individualized views.

Unified Index to Information Repositories

Amount of information is doubling every 18 months, and unstructured information volumes grow six times faster than structured.

Employees spend far too much time, about 20% of their time, on average, looking for, not finding and recreating information. Once they find the information, 42% of employees report having used the wrong information, according to a recent survey.

To combat this reality, for years, companies have spent hundreds of thousands, even millions, to move data to centralized systems, in an effort to better manage and access its growing volumes, only to be disappointed as data continues to proliferate outside of that system. Even with a single knowledgebase in place, employees report decrease in critical customer service metrics, due to the inability to quickly locate the right knowledge and information to serve customers.

Despite best efforts to move data to centralized platforms, companies are finding that their knowledgebase runs throughout enterprise systems, departments, divisions and newly acquired subsidiaries. Knowledge is stored offline in PCs and laptops, in emails and archives, intranets, file shares, CRM systems, ERPs, home-grown systems, and many others—across departments and across geographies.

Add to this the proliferation of enterprise applications use (including social networks, wikis, blogs and more) throughout organizations and it is no wonder that efforts to consolidate data into a single knowledgebase, a single "version of the truth" have failed... and at a very high price.

The bottom line is, moving data into a single knowledgebase is a losing battle. There remains a much more successful way to effectively manage your knowledge ecosystem without moving data.

When there are multiple systems containing organization's information are in place, a better approach is to stop moving data by combining structured and unstructured data from virtually any enterprise system, including social networks, into a central, unified index. Think of it as an indexing layer that sits above all enterprise systems, from which services can be provided to multiple departments, each configured to that department’s specific needs.

This approach enables dashboards, focused on various business departments and processes, prospective customers. Such composite views of information provide new, actionable perspectives on many business processes, including overall corporate governance. The resulting juxtaposition of key metrics and information improves decision making and operational efficiency.

This approach allows IT departments to leverage their existing technology, and avoid significant cost associated with system integration and data migration projects. It also helps companies avoid pushing their processes into a one-size-fits-all, cookie-cutter framework.

With configurable dashboards, companies decide how/what/where information and knowledge is presented, workflows are enabled, and for what groups of employees. Information monitoring and alerts facilitate compliance. There is virtually no limit to the type of information and where it is pulled from, into the central, unified and, importantly, highly secure index: structured, unstructured, from all corporate email, files, archives, on desktops and in many CRMs, CMS, knowledgebases, etc.

Enterprise applications have proliferated throughout organizations, becoming rich with content. And yet all of that knowledge and all of that content remain locked within the community, often not even easily available to the members themselves.

Now it is possible to leverage the wisdom of communities in enterprise search efforts. User rankings, best bets and the ability to find people through the content they create are social search elements that provide the context employees and customers have come to expect from their interactions with online networks.

Imagine one of your sales executives attempting to sell one of your company’s largest accounts. They access a composite, 360 degree view of that company, and see not only the account history, sales opportunities, contact details, prior email conversations, proposals, contracts, customer service tickets, that customer’s recent comments to a blog post, complaints about service or questions posed within your customer community.

Armed with this knowledge, your sales executive is in a more informed position to better assist and sell to that customer. Without moving data your sales executive has a single, composite view of information that strategically informs the sales process.

Ubiquitous knowledge access allows employees to search where they work. Once you created the central index, you need to provide your employees with anytime/anywhere access to pertinent information and knowledge.

In many organizations, employees spend a lot of their time in MS Outlook. Other organizations with large sales teams need easy access to information on the road. Also valuable is the ability to conduct secure searches within enterprise content directly from a BlackBerry, including guided navigation. Even when systems are disconnected, including laptops, users can easily find information from these systems, directly from their mobile device. Again, without moving data, organizations can enjoy immediate, instant access to pertinent knowledge and information, anywhere, anytime.

Companies that stopped moving data report favorable results of their unified information index layer from multiple repositories such as faster customer issues resolution time, significant reduction in dedicated support resources, savings in upgrade cost for the legacy system which was replaced, increase in self-service customer satisfaction, and reducing average response time to customers' queries.

There are few applications currently in the market that fulfill these functions. These are enterprise search applications.

However, there is no "one fits all" approach. Any solution should be based on organization's business requirements.