Case Study: Building crosswalks between taxonomies

Introduction

Mapping two taxonomies involves aligning and integrating two distinct classification systems, where each system categorizes and organizes data or concepts according to specific rules and criteria. This process is crucial for businesses because it enables interoperability between different systems, allowing for seamless data integration, improved data accuracy, and more effective cross-functional analysis. The importance of taxonomy mapping lies in its ability to unify disparate data sources, making them accessible and usable across various platforms and departments.

However, the complexity of modern data environments poses significant challenges for taxonomy mapping. These challenges include dealing with inconsistent terminologies, gaps in one or both taxonomies, varying levels of granularity between taxonomies, and the dynamic nature of data, where taxonomies frequently evolve. Additionally, the lack of standardized frameworks and the increasing volume of data further complicate the mapping process, requiring sophisticated algorithms and substantial manual effort to ensure precise alignment and avoid semantic discrepancies.

Historical Challenges with Taxonomy Mapping

Historically, building crosswalks between two taxonomies together typically involved a combination of both manual and algorithmic efforts, including:

• Mapping Tables: Subject matter experts analyze the taxonomies and manually align categories, terms, and concepts. Creating mapping tables or spreadsheets where corresponding terms from each taxonomy are matched. This approach is straightforward but time-consuming and prone to human error.
• Ontology Matching Algorithms: Software tools that use algorithms to automatically identify relationships between terms in different taxonomies. These tools often rely on natural language processing (NLP), machine learning, and semantic analysis to suggest mappings. However, this approach may not capture the nuances of the taxonomies.
• Rule-Based Systems: Systems that apply predefined rules or logic to map terms between taxonomies. These rules are often created based on patterns observed in the taxonomies or industry-specific knowledge. While rule-based systems are effective for simple low-complexity taxonomies, they lack the robustness to handle complex mappings.
• Business Feedback Loops: Continuous refinement of mappings based on feedback from users or errors encountered in real-world applications. This iterative process helps in improving the accuracy and relevance of the mapping over time. However, this can be costly and introduce risks if not managed effectively.
• Industry Standards: Industries or organizations often adopt standardized taxonomies, reducing the need for complex mapping. However, standardized taxonomies rarely cover all use cases, so mapping a standard taxonomy to an internal or secondary one is often required.

Despite these methods, building accurate, comprehensive crosswalks between taxonomies remains a challenging task, especially when dealing with large, complex, or evolving datasets and taxonomies.

Automatic Taxonomy Mapping

Taylor is excited to announce breakthrough research in automatic taxonomy mapping, a cutting-edge solution that leverages advanced AI and machine learning techniques to streamline the process of aligning taxonomies. Our approach combines the power of deep learning, natural language processing, and semantic analysis to automatically align categories, terms, and concepts across different taxonomies with high accuracy and efficiency. By automating the mapping process, Taylor's solution significantly reduces the time and effort required to build crosswalks, enabling businesses to achieve seamless data integration and interoperability across systems.

Taylor's methodology for automatic taxonomy mapping:

Data Preprocessing

• Text Normalization: Taylor normalizes the terms within each taxonomy. This includes lowercasing, removing punctuation, stopword removal, and lemmatization. These help avoid discrepancies.
• Tokenization: Tokenize the terms into individual words or meaningful phrases. This step is crucial for subsequent matching steps.

Semantic Analysis

• Synonym Expansion: Taylor uses domain-specific thesauri to expand each term into its synonyms. This helps in identifying equivalent terms that may not be lexically identical.
• Embedding Representations: Taylor generates vector representations of the terms.
• Contextual Similarity: If taxonomies include terms that are context-dependent, we use contextual embeddings to capture the meaning of terms in different contexts. This is useful for polysemous words.

Initial Matching

• String Matching: Taylor performs basic string matching on the tokenized and normalized terms.
• Embedding Similarity Calculation: Taylor calculates and filters the cosine similarity between the vector embeddings of terms from the two taxonomies. Taylor tunes the threshold relatively to filter out weak matches.

Machine Learning and Rule-Based Refinement

• Unsupervised Learning: Taylor clusters similar terms and potential mappings.
• Rule-Based Enhancements: Domain-specific rules are applied to capture the nuances of the taxonomies. For instance, in the medical domain, certain prefixes or suffixes may have specific meanings that are important for accurate mapping.

Validation and Feedback Loop

• Confidence Scoring: Taylor assigns confidence scores to each mapping based on the strength of the match. This helps in prioritizing mappings for manual review or correction.
• User Feedback: Taylor provides a UI for business users to incorporate feedback from users or domain experts to validate and refine the mappings. This feedback loop ensures that the mappings are accurate and relevant to the specific use case.

Integration and Deployment

• To work with your mappings, Taylor's API exposes the relationships between taxonomies. The API allows for querying mappings, adding new mappings, retrieving mapping confidence scores, and triggering new mappings on new taxonomies.

Continuous Monitoring and Updating

• Monitoring: Taylor's monitoring systems track the performance of the mapping system in production.
• Dynamic Adaptation: As taxonomies evolve or new versions of taxonomies are released, Taylor's systems are robust and accommodate new terms or structures. For example, when a new version of a taxonomy is released, Taylor can automatically adapt the mappings to the new version.

For Business & Product Managers

Business and product teams can leverage the automatic taxonomy mapping solution to streamline data integration, improve data quality, and enhance cross-functional collaboration by logging in to the Taylor platform. Taylor's user-friendly interface allows users to upload taxonomies, view suggested mappings, and edit mappings.

For Developers

Developers can integrate Taylor's automatic taxonomy mapping solution into their applications using the Taylor API. The API provides endpoints for uploading taxonomies, retrieving mappings, and validating mappings. Developers can also leverage the API to trigger new mappings, monitor mapping performance, and receive real-time updates on mapping changes.

Note: The Taylor Mapping API is currently available to select partners. If you are interested in integrating the automatic taxonomy mapping solution into your application, please contact us for more information.

Integrations & Deployment

Sign in here to start mapping your taxonomies with Taylor's automatic taxonomy mapping solution.