{
  "version": 3,
  "sources": ["ssg:https://framerusercontent.com/modules/YVgynkVt96a0ES9Gk8yz/VEx2WPNLcsYT7stAmBDF/eo4RAmtig-19.js"],
  "sourcesContent": ["import{jsx as e,jsxs as t}from\"react/jsx-runtime\";import{Link as a}from\"framer\";import{motion as n}from\"framer-motion\";import*as i from\"react\";export const richText=/*#__PURE__*/t(i.Fragment,{children:[/*#__PURE__*/e(\"p\",{children:\"As AI advances across industries, LLMs remain key catalysts of its progress. Practitioners are developing countless LLMs for various applications in various domains. But with so many models available, how do we choose the right one for our use case?\"}),/*#__PURE__*/t(\"p\",{children:[\"The answer is \",/*#__PURE__*/e(\"strong\",{children:\"LLM leaderboards.\"})]}),/*#__PURE__*/e(\"h2\",{children:\"What is an LLM leaderboard?\"}),/*#__PURE__*/e(\"p\",{children:\"An LLM leaderboard is a ranking platform that compares the performances of different large language models. It is a contest-like environment where models are ranked for tasks like code generation, question answering, summarization, and general understanding.\"}),/*#__PURE__*/e(\"p\",{children:\"LLM leaderboards are essential because they aim to provide unbiased and uncontaminated results by comparing models side-by-side using predefined criteria. These leaderboards help users and developers learn which model is right for them through transparent and objective model evaluations.\"}),/*#__PURE__*/e(\"p\",{children:\"There are different types of leaderboards. Some assess general LLM skills, such as how relevant and coherent their responses are and whether they follow instructions. Others focus on domain-specific LLMs and their performance in specialized tasks pertinent to the field. Several platforms prioritize the practical use of LLMs and rank them based on speed and cost.\"}),/*#__PURE__*/e(\"p\",{children:\"Leaderboards also address issues like hallucinations, bias, and toxicity, much-needed factors for ensuring responsible use of LLMs.\"}),/*#__PURE__*/e(\"p\",{children:\"This article explores how leaderboards test and rank models, along with examples of the most popular ones.\"}),/*#__PURE__*/e(\"h2\",{children:\"How to decide which model is the best?\\xa0\"}),/*#__PURE__*/e(\"p\",{children:\"Leaderboards use different methods to grade LLMs, which generally include:\"}),/*#__PURE__*/t(\"ul\",{children:[/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",style:{\"--framer-font-size\":\"11px\",\"--framer-text-color\":\"rgb(0, 0, 0)\",\"--framer-text-decoration\":\"none\"},children:/*#__PURE__*/e(\"p\",{children:'Quantitative metrics, usually accompanied by a minimum threshold that defines if a model is \"good\" at the task in hand or,'})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",style:{\"--framer-font-size\":\"11px\",\"--framer-text-color\":\"rgb(0, 0, 0)\",\"--framer-text-decoration\":\"none\"},children:/*#__PURE__*/e(\"p\",{children:\"A relative comparison against other models using human evaluation.\\xa0\"})})]}),/*#__PURE__*/e(\"p\",{children:\"A leaderboard tests the models using standardized criteria. Comprehensive evaluations ensure a fair comparison and help users learn which models achieve higher accuracies, lower costs, and turbo speeds. The criteria can vary from task to task based on required capabilities, price point, scale, and field of use. Most leaderboards include metrics that analyze these aspects in some way. Let's discuss some of the most common ones.\"}),/*#__PURE__*/t(\"ul\",{children:[/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",style:{\"--framer-font-size\":\"11px\",\"--framer-text-color\":\"rgb(0, 0, 0)\",\"--framer-text-decoration\":\"none\"},children:/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Accuracy\"}),\" is a simple but key concept. In tasks like classification, we want to know if the model produces the correct outputs and use accuracy to measure its success. \",/*#__PURE__*/e(\"strong\",{children:\"F1 score\"}),\" is often used as an alternative to accuracy. It tells us how well the model balances precision and recall, which is especially valuable in imbalanced classification problems.\"]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",style:{\"--framer-font-size\":\"11px\",\"--framer-text-color\":\"rgb(0, 0, 0)\",\"--framer-text-decoration\":\"none\"},children:/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Speed-related metrics\"}),\" are crucial for real-time applications like chat. Therefore, model ranking platforms usually include data on how quickly a model responds or how many tokens it generates per second.\"]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",style:{\"--framer-font-size\":\"11px\",\"--framer-text-color\":\"rgb(0, 0, 0)\",\"--framer-text-decoration\":\"none\"},children:/*#__PURE__*/t(\"p\",{children:[\"Then there's \",/*#__PURE__*/e(\"strong\",{children:\"perplexity\"}),\", a score that shows a model's predictive capabilities. A lower perplexity score means the model is better at generating coherent and logically structured text, essential for tasks like text generation, machine translation, and summarization.\"]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",style:{\"--framer-font-size\":\"11px\",\"--framer-text-color\":\"rgb(0, 0, 0)\",\"--framer-text-decoration\":\"none\"},children:/*#__PURE__*/t(\"p\",{children:[\"Other metrics measure contextual relevance and semantic understanding. For example, the \",/*#__PURE__*/e(\"strong\",{children:\"BERT score\"}),\" indicates how accurately the model represents the semantics of a reference text. Such evaluations are commonly used in text summarization or translation tasks.\"]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",style:{\"--framer-font-size\":\"11px\",\"--framer-text-color\":\"rgb(0, 0, 0)\",\"--framer-text-decoration\":\"none\"},children:/*#__PURE__*/e(\"p\",{children:\"Ethical and responsible AI is getting more attention and requires metrics addressing this aspect of LLMs. These metrics are crucial for real-world applications to ensure the model complies with law regulations and won't provide unethical content.\\xa0\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",style:{\"--framer-font-size\":\"11px\",\"--framer-text-color\":\"rgb(0, 0, 0)\",\"--framer-text-decoration\":\"none\"},children:/*#__PURE__*/t(\"p\",{children:[\"Specific leaderboards also evaluate a \",/*#__PURE__*/e(\"strong\",{children:\"model's likelihood of hallucinating\"}),\" and whether its output is correct and based on a verifiable source.\"]})})]}),/*#__PURE__*/e(\"p\",{children:\"These are some frequently used metrics, often combined in an overall score of a model's abilities.\\xa0\"}),/*#__PURE__*/e(\"p\",{children:\"The choice of metrics depends on what we seek in a model. Still, regardless of the use case, metrics must accurately assess the required capabilities of the LLMs while still being:\\xa0\"}),/*#__PURE__*/t(\"ul\",{children:[/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",style:{\"--framer-font-size\":\"11px\",\"--framer-text-color\":\"rgb(0, 0, 0)\",\"--framer-text-decoration\":\"none\"},children:/*#__PURE__*/e(\"p\",{children:\"Simple enough to clearly understand what they measure,\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",style:{\"--framer-font-size\":\"11px\",\"--framer-text-color\":\"rgb(0, 0, 0)\",\"--framer-text-decoration\":\"none\"},children:/*#__PURE__*/e(\"p\",{children:\"Reliable to ensure consistency and invariant outcomes and\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",style:{\"--framer-font-size\":\"11px\",\"--framer-text-color\":\"rgb(0, 0, 0)\",\"--framer-text-decoration\":\"none\"},children:/*#__PURE__*/e(\"p\",{children:\"Relevant to the model's purpose.\"})})]}),/*#__PURE__*/e(\"h2\",{children:\"How do we make sure the results are objective?\"}),/*#__PURE__*/e(\"p\",{children:\"As many LLMs differ in nuances, selecting the best model can be challenging.\"}),/*#__PURE__*/e(\"p\",{children:\"To minimize bias and assure fairness, we evaluate the models against designated standards - benchmarks. These consist of problems, questions, and tasks with ground-truth answers. The LLMs' outputs are scored based on their similarity to the expected answer. Depending on the benchmark, we can assess the model's proficiency in code generation, problem-solving, and even emotional intelligence. Benchmarks are essential for fair comparison because they define clear and consistent rules for measuring performance, scale, and capabilities. Some benchmarks have established their place in the community for their holistic approach to measuring LLM performance. Let's look at some of the most popular benchmarks at the moment.\"}),/*#__PURE__*/t(\"ul\",{children:[/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",style:{\"--framer-font-size\":\"11px\",\"--framer-text-color\":\"rgb(0, 0, 0)\",\"--framer-text-decoration\":\"none\"},children:/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(a,{href:\"https://eqbench.com/\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(n.a,{children:\"EQ-Bench\"})}),\" tests the emotional intelligence of LLMs, which is still relatively overlooked on other platforms. This benchmark evaluates the models' ability to interpret social interactions by asking them how intense emotions are in different dialogues. The benchmark even strongly connects to other multi-domain datasets like\",/*#__PURE__*/e(a,{href:\"https://paperswithcode.com/dataset/mmlu\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(n.a,{children:\" MMLU\"})}),\", suggesting it evaluates similar aspects of general intelligence.\"]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",style:{\"--framer-font-size\":\"11px\",\"--framer-text-color\":\"rgb(0, 0, 0)\",\"--framer-text-decoration\":\"none\"},children:/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(a,{href:\"https://huggingface.co/datasets/openai/openai_humaneval\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(n.a,{children:\"HumanEval\"})}),\" is another well-known benchmark that assesses how accurately models generate code from docstrings. Practitioners utilize it to determine a model's programming knowledge and general approach to problem-solving.\"]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",style:{\"--framer-font-size\":\"11px\",\"--framer-text-color\":\"rgb(0, 0, 0)\",\"--framer-text-decoration\":\"none\"},children:/*#__PURE__*/t(\"p\",{children:[\"Measuring Massive Multitask Language Understanding (\",/*#__PURE__*/e(a,{href:\"https://paperswithcode.com/dataset/mmlu\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(n.a,{children:\"MMLU\"})}),\") is one of the most comprehensive benchmarks for LLM evaluations.\",/*#__PURE__*/e(a,{href:\"https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(n.a,{children:\" The HuggingFace Open LLM leaderboard\"})}),\" uses MMLU to demonstrate how LLMs perform in different areas, including law and computer science. LLMs that score high on this benchmark are skilled at multitasking and reasoning across various domains.\"]})})]}),/*#__PURE__*/t(\"h5\",{children:[\"Toloka is developing \",/*#__PURE__*/e(a,{href:\"https://toloka.ai/blog/toloka-s-advanced-benchmarking-reveals-limitations-and-opportunities-for-llms/\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(n.a,{children:\"benchmarks\"})}),\" for specialized domains like natural sciences and university-level math to reveal limitations and opportunities for LLMs.\\xa0\"]}),/*#__PURE__*/e(\"h2\",{children:\"Which are some top-caliber leaderboards?\"}),/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(a,{href:\"https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(n.a,{children:\"The Open LLM Leaderboard\"})}),\",\",/*#__PURE__*/e(a,{href:\"https://lmarena.ai/?leaderboard\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(n.a,{children:\" LMSYS Chatbot Arena\"})}),\", and\",/*#__PURE__*/e(a,{href:\"https://artificialanalysis.ai/leaderboards/models\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(n.a,{children:\" Artificial Analysis LLM Leaderboard\"})}),\" are among the leading LLM leaderboards due to their robust and innovative evaluation methods. These platforms use comprehensive datasets with real-world tasks and a mix of human and automated assessments to provide transparent and reliable rankings. Let's have a look at their evaluation target and methodology:\"]}),/*#__PURE__*/t(\"ul\",{children:[/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",style:{\"--framer-font-size\":\"11px\",\"--framer-text-color\":\"rgb(0, 0, 0)\",\"--framer-text-decoration\":\"none\"},children:/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(a,{href:\"https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(n.a,{children:\"The Open LLM Leaderboard\"})}),\" by Hugging Face is an established resource for insights into LLMs' problem-solving skills and commonsense reasoning. The leaderboard uses six Eleuther AI LM Evaluation Harness benchmarks to test LLMs on math, physics, and general knowledge. It even reveals the environmental impact by reporting data on the C2O emissions from the models' evaluations. This leaderboard is popular among developers as it allows anyone to evaluate their model and see how it competes with existing open-source models.\"]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",style:{\"--framer-font-size\":\"11px\",\"--framer-text-color\":\"rgb(0, 0, 0)\",\"--framer-text-decoration\":\"none\"},children:/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(a,{href:\"https://lmarena.ai/?leaderboard\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(n.a,{children:\"LMSYS Chatbot Arena\"})}),\" is a model ranking platform with a more hands-on experience. It creates a contest-like environment where the models are ranked based on the popular Elo rating system used in chess. On this crowdsourced platform, users compare responses from two models to the same prompt and vote for the better one. They do this iteratively until a winner is decided. This leaderboard relies on human input to rank models in conversational tasks, reasoning, and context handling.\"]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",style:{\"--framer-font-size\":\"11px\",\"--framer-text-color\":\"rgb(0, 0, 0)\",\"--framer-text-decoration\":\"none\"},children:/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(a,{href:\"https://artificialanalysis.ai/leaderboards/models\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(n.a,{children:\"The Artificial Analysis LLM Leaderboard\"})}),\" compares over 30 models, including GPT-4o, Gemini, Llama 3, and GPT-3.5 Turbo, on diverse metrics such as price, latency, and context window. The\",/*#__PURE__*/e(a,{href:\"https://artificialanalysis.ai/leaderboards/providers\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(n.a,{children:\" LLM API Providers Leaderboard\"})}),\" compares over 100 LLM endpoints on the same metrics. These platforms help customers and users choose the right AI model and hosting provider for their use cases.\"]})})]}),/*#__PURE__*/t(\"p\",{children:[\"In addition to the above-listed leaderboards, there are several other honorable mentions. For example, the\",/*#__PURE__*/e(a,{href:\"https://tatsu-lab.github.io/alpaca_eval/\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(n.a,{children:\" AlpacaEval Leaderboard\"})}),\" assesses the LLMs' ability to follow user instructions. The\",/*#__PURE__*/e(a,{href:\"https://huggingface.co/spaces/mteb/leaderboard\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(n.a,{children:\" MTEB\"})}),\" leaderboard showcases the top text-embedding models based on the Massive Text Embedding Benchmark. As hallucinations remain a weak spot for LLMs, leaderboards like\",/*#__PURE__*/e(a,{href:\"https://huggingface.co/spaces/vectara/leaderboard\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(n.a,{children:\" HHEM\"})}),\" try to measure their resilience against generating false information. LLMs have become prevalent in programming and automation, and practitioners rely on platforms like\",/*#__PURE__*/e(a,{href:\"https://huggingface.co/spaces/mike-ravkine/can-ai-code-results\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(n.a,{children:\" CanAI Code\"})}),\",\",/*#__PURE__*/e(a,{href:\"https://evalplus.github.io/leaderboard.html\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(n.a,{children:\" EvalPlus\"})}),\", and\",/*#__PURE__*/e(a,{href:\"https://gorilla.cs.berkeley.edu/leaderboard.html\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(n.a,{children:\" Berkley Function-Calling Leaderboard\"})}),\" to find the top-tier programming models in the field.\\xa0\"]}),/*#__PURE__*/e(\"h2\",{children:\"Key takeaways\"}),/*#__PURE__*/e(\"p\",{children:\"Leaderboards are the best way to stay informed about the rapidly evolving field of LLMs. They provide equal ground for evaluating models. A good leaderboard is fair, reliable, and practical. It utilizes standardized benchmarks and metrics that are easy to understand and focus on tasks applicable to real-world scenarios.\"}),/*#__PURE__*/e(\"p\",{children:\"Another essential characteristic is transparency. These platforms clearly explain how models are evaluated and make it easy to reproduce results.\"}),/*#__PURE__*/e(\"p\",{children:\"They also encourage innovation by presenting challenging problems for models to solve. A well-designed leaderboard is inclusive and accessible to participants from eclectic backgrounds. It remains relevant by consistently tracking model progress and updating evaluation methods.\"}),/*#__PURE__*/e(\"h2\",{children:\"Learn more\"}),/*#__PURE__*/t(\"p\",{children:[\"Looking for guidance on LLM evaluation? Our experts are here to help.\\xa0 \\xa0 \\xa0 \\xa0 \\xa0 \\xa0 \\xa0 \\xa0 \\xa0 \\xa0 \\xa0 \\xa0 \\xa0 \\xa0 \\xa0 \\xa0 \\xa0 \",/*#__PURE__*/e(a,{href:\"https://toloka.ai/talk-to-us\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(n.a,{children:\"Contact us\"})}),\" to get started.\"]})]});export const richText1=/*#__PURE__*/t(i.Fragment,{children:[/*#__PURE__*/e(\"p\",{children:\"Envision the exhilaration of a shopper when their product search unlocks the perfect find \u2013 this is the realm of retail where Turkish Hepsiburada aimed for nothing short of excellence. To revolutionize its search capabilities for the millions of products available on its e-commerce platform, Turkish Hepsiburada teamed up with Toloka and developed a solution based on Microsoft Azure. The result was a 9-percent increase in search quality. Hepsiburada is now one of the leading Turkish e-commerce platforms. \"}),/*#__PURE__*/e(\"p\",{children:'In 1998, five visionaries began Hepsiburada in a humble studio flat, aiming to supply the growing demands of the information technology (IT) hardware market. Today, Hepsiburada stands as one of the largest, fastest growing and leading e-commerce platforms in T\\xfcrkiye. Hepsiburada\u2019s platform, which began with a handful of tech products, now boasts over 195 million stock keeping units (SKUs) from more than 101,000 vendors. \"About 12 million active customers access to our vast product range monthly through our application. This application provides not only a marketplace, but also financial services, express delivery, ticketing, and more.\" shares Alexey Shevenkov, Chief Technology Officer at Hepsiburada.'}),/*#__PURE__*/t(\"blockquote\",{children:[/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"Imagine yourself searching for a product and instantly finding it. Not just any product, but the right one which is also trusted and popular with other shoppers. That's the seamless experience we aim to offer at Hepsiburada.\"})}),/*#__PURE__*/e(\"p\",{children:\"\u2014 Alexey Shevenkov, CTO, Hepsiburada\"})]}),/*#__PURE__*/e(\"p\",{children:'\"We had an ambitious goal to provide our customers the best online shopping experience ever,\" shares Shevenkov. But the effectiveness of search quality can make or break the retail experience. When a customer looks for a product and it isn\\'t available, that\u2019s disappointing. When the product is available, but not easily found due to subpar search results, that frustrates a customer. So Shevenkov knew that goal wouldn\u2019t be realized without truly understanding the behavior of users on the platform. \"Imagine yourself searching for a product and instantly finding it. Not just any product, but the right one which is also trusted and popular with other shoppers. That\\'s the seamless experience we aim to offer at Hepsiburada,\" he says.'}),/*#__PURE__*/e(\"p\",{children:\"Recognizing that when it comes to navigating the intricate web of e-commerce, two heads \u2014 or in this case, companies \u2014 are better than one, Hepsiburada sought the expertise of Toloka, a technology trailblazer known for its expertise in enhancing machine learning processes with human insights.\"}),/*#__PURE__*/e(\"h2\",{children:\"Revolutionizing retail search together\"}),/*#__PURE__*/e(\"p\",{children:'To leverage the direct relationship between search quality and conversion rates, Toloka helped Hepsiburada design a tool to regularly evaluate search performance and measure a wider range of metrics. \"Relying on metrics like quantity of orders and clicks are limited to only identifying things like popular products. We wanted additional metrics that would enable us to know what customers need and offer that to them even before they click on something,\" explains Shevenkov.'}),/*#__PURE__*/e(\"p\",{children:'To enhance conversions from searches, Toloka proposed to construct search metrics, which would not only train their machine learning models in the future but also routinely assess search performance. \"Our method considers not just standard metrics but also customer preferences, knowledge, requirements, product popularity, and more, to determine the most efficient search system,\" says Olga Megorskaya, CEO at Toloka.'}),/*#__PURE__*/e(\"p\",{children:'Toloka\u2019s platform runs on Microsoft Azure. \"All our labeling, models, demos, and related tasks run through Azure\u2014without it, none of this would be possible,\" shares Megorskaya.'}),/*#__PURE__*/t(\"blockquote\",{children:[/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"All our labeling, models, demos, and related tasks run through Azure \u2014 without it, none of this would be possible.\"})}),/*#__PURE__*/e(\"p\",{children:\"\u2014 Olga Megorskaya, CEO, Toloka\"})]}),/*#__PURE__*/e(\"p\",{children:\"By blending online user behavior data with offline labeling signals, Hepsiburada can discern genuine user intent beyond just clicks. It implemented the solution across their marketplace to support quick shipping, vertical browsing, and other features.\"}),/*#__PURE__*/e(\"h2\",{children:\"Better search results leading to higher conversion rate\"}),/*#__PURE__*/e(\"p\",{children:'The search experience has improved for customers with more accurate results. Hepsiburada meticulously identified and resolved common e-commerce issues like incorrect product sorting and incorrect category predictions, both of which account for 8.3 percent and 15 percent of search failures respectively. \"We\u2019ve achieved a 9-percent increase in search quality. This, in turn, has boosted the search conversion rate,\" Shevenkov proudly shares.'}),/*#__PURE__*/e(\"p\",{children:'Another tool Hepsiburada deployed is competitor comparison. Through crowdsourcing, Hepsiburada and Toloka labeled search results for random queries. \"This allows us to gauge our search quality vis-\\xe0-vis our competitors,\" elaborates Shevenkov. \"We\\'ve already observed an 8 percent improvement in measured relevance, which correlates with increase in sales and customer satisfaction.\"'}),/*#__PURE__*/e(\"p\",{children:'As the journey from search to purchase becomes more seamless, Hepsiburada keeps thinking of ways to enhance the customer experience. \"In the realm of e-commerce, metrics play a pivotal role. By utilizing the right metrics and collaborating with the right partners, retailers can embark on a cycle of constant improvement. Then, for the shopper, we deliver an even better, seamless shopping journey,\" concludes Shevenkov.'}),/*#__PURE__*/t(\"blockquote\",{children:[/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"We\u2019ve achieved a 9-percent increase in search quality. This, in turn, has boosted the search conversion rate.\"})}),/*#__PURE__*/e(\"p\",{children:\"\u2014 Alexey Shevenkov, CTO, Hepsiburada\"})]}),/*#__PURE__*/t(\"p\",{children:[\"The content of this article was originally posted on \",/*#__PURE__*/e(a,{href:\"https://customers.microsoft.com/en-us/story/1688738181933208375-hepsiburada-retailers-azure-en-turkiye\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(n.a,{children:\"Microsoft Customer Stories\"})}),\".\"]})]});export const richText2=/*#__PURE__*/t(i.Fragment,{children:[/*#__PURE__*/e(\"p\",{children:\"Imagine a world where artificial intelligence isn't confined to a single mode of interaction but can seamlessly bridge the gap between text and images, language and vision, and even sound and sensation. AI is no longer limited to just processing text or images but can understand, generate, and interpret information across multiple domains. That\u2019s where the concept of foundation models steps in.\"}),/*#__PURE__*/t(\"p\",{children:[\"These models, often associated with \",/*#__PURE__*/e(a,{href:\"https://toloka.ai/large-language-models/\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(n.a,{children:\"large language models\"})}),\", have evolved to encompass a wide array of applications, proving their versatility beyond just text processing. In this article, we will explore the evolving landscape of foundation models, their diverse capabilities, and the impact they have on the AI industry by unlocking the power of multimodality.\"]}),/*#__PURE__*/e(\"h2\",{children:\"What is a Foundation Model\"}),/*#__PURE__*/t(\"p\",{children:[\"A foundation model is a large, pre-trained neural network designed to understand and generate various types of content. These models are trained on massive datasets, which allows them to capture the intricacies and nuances of data. A group of data scientists at Stanford Institute for Human-Centered Artificial Intelligence (HAI) Center for Research on Foundation Models came up with the term in research on foundation models called \",/*#__PURE__*/e(a,{href:\"https://arxiv.org/abs/2108.07258\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(n.a,{children:\"On the Opportunities and Risks of Foundation Models\"})}),'. They referred to it as \u201Cany model that is trained on broad data (generally using self-supervision at scale) that can be adapted (e.g., fine-tuned) to a wide range of downstream tasks\".']}),/*#__PURE__*/e(\"h3\",{children:\"Pretraining and Fine-Tuning of Foundation Models\"}),/*#__PURE__*/t(\"p\",{children:[\"As indicated in the definition above, foundation models are primarily based on \",/*#__PURE__*/e(\"strong\",{children:\"self-supervised learning (SSL)\"}),\". It is a powerful approach to AI systems training that allows data scientists to leverage huge data sets reducing or eliminating time-consuming labeling. During SSL model learns to generate its own labels from the input data, eliminating the need for external labels.\"]}),/*#__PURE__*/t(\"p\",{children:[\"Foundation models are usually \",/*#__PURE__*/e(\"strong\",{children:\"pre-trained\"}),\" on large, diverse datasets, such as texts taken from various websites. Pretraining is a machine learning technique in which a model is trained on a large dataset to learn general features of training data before being fine-tuned for a specific task.\"]}),/*#__PURE__*/e(\"p\",{children:\"Pre-training begins with the collection of a vast and diverse dataset that is relevant to the domain the model is intended for. For text-based foundation models, this dataset often includes extensive text data from sources like books, articles, websites, and other textual sources. For multimodal models, the dataset may consist of text, images, audio, and other types of data.\"}),/*#__PURE__*/e(\"p\",{children:\"In self-supervised learning, the primary idea is to train a model using unlabeled data and generate tasks similar to the ones presented in supervised learning from that unlabeled data without relying on external labels. However, there may be situations where labeled data is used in conjunction with self-supervised learning, but the labeled data is typically introduced during the fine-tuning phase or for specific downstream tasks.\"}),/*#__PURE__*/t(\"p\",{children:[\"Pretrained model can be \",/*#__PURE__*/e(\"strong\",{children:\"fine-tuned\"}),\" with the right data to make them serve their intended purpose. Such foundation models may then serve as the basis or foundation for many AI applications, hence their name. This fine-tuning step involves adjusting the model's parameters to perform well on the task at hand. The general knowledge and features learned during pre training are adapted for the specific task that is typically supervised learning, because labeled data is introduced during that stage.\"]}),/*#__PURE__*/e(\"h3\",{children:\"Role of Deep Learning in Foundation Models\"}),/*#__PURE__*/t(\"p\",{children:[\"The concept of the foundation model is also based on \",/*#__PURE__*/e(\"strong\",{children:\"deep learning\"}),\". It is a subfield of machine learning that leverages neural networks with multiple layers to process and learn from vast datasets. Deep learning models can scale to handle vast amounts of data. This scalability is essential for foundation models, allowing them to process and learn from large and diverse datasets.\"]}),/*#__PURE__*/e(\"p\",{children:\"This architecture is the driving force behind foundation models and enables them to excel in understanding and generating information. These models use deep learning techniques to learn complex patterns, hierarchies, and representations of data, making them adaptable to diverse tasks.\"}),/*#__PURE__*/e(\"p\",{children:\"Deep learning provides the flexibility and adaptability to integrate different modalities into a single model. Foundation models therefore not only can be applied to a range of natural language processing (NLP) tasks but also can handle multiple modalities, such as text and images or text, images, video, and audio.\"}),/*#__PURE__*/e(\"p\",{children:\"A deep neural network consists of multiple layers of interconnected neurons that can be fine-tuned to process different types of data, including text, images, audio, and more. This adaptability allows deep foundation models to be versatile enough to handle multimodal input. Their multimodal capability is a key feature that distinguishes them from specialized models designed for single tasks.\"}),/*#__PURE__*/e(\"p\",{children:\"The thing is, traditional machine learning models, while capable of solving specific tasks, often lack the massive scale and capacity of foundation models. They are designed for specific tasks and may not be able to apply well to diverse applications.\"}),/*#__PURE__*/e(\"p\",{children:\"To summarize all of the above, the Foundation models represent a particular type of multimodal AI model that combines pre-training through self-supervision, deep learning architecture and the ability to be fine-tuned later. Such models have started to rapidly evolve in the last decade. Therefore, specialists at the Stanford Center for Research on Foundation Models decided to give this new promising tendency a name - foundation models. Let's take a closer look at other prominent features of foundation models that can help us distinguish them from all other types of AI models.\"}),/*#__PURE__*/e(\"h2\",{children:\"Distinguishing Characteristics of Foundation Models\"}),/*#__PURE__*/e(\"p\",{children:\"In addition to the characteristics above mentioned, foundation models possess several distinctive features that set them apart from other types of AI models. These characteristics make them versatile and adaptable, allowing them to serve as a foundation upon which various specialized AI applications can be built. Here are the features that distinguish foundation models from other AI models.\"}),/*#__PURE__*/e(\"h3\",{children:\"Transfer Learning\"}),/*#__PURE__*/e(\"p\",{children:\"Foundation models are designed with the idea of transfer learning. It is a machine learning technique where a model trained on one task or dataset is adapted to perform another related task. The idea behind transfer learning is to leverage the knowledge gained during the initial training on one task to improve performance on a new, but related, task.\"}),/*#__PURE__*/e(\"p\",{children:\"They are pre-trained on a large corpus of text or data, making them adaptable to various tasks and domains. This transfer learning approach reduces the need for training models from scratch for specific applications.\"}),/*#__PURE__*/e(\"p\",{children:\"After pre-training, foundation models can be fine-tuned for certain tasks, thanks to the concept of transfer learning. This adaptability sets them apart from models that are solely designed for one specific purpose. Fine-tuning allows foundation models to excel in a wide range of tasks, such as language translation, image classification, and more.\"}),/*#__PURE__*/e(\"h3\",{children:\"Transformer Architecture\"}),/*#__PURE__*/e(\"p\",{children:'Foundation models are typically (not mandatorily) based on transformer architecture, which now has become the standard for many natural language processing and multimodal models. Transformers are a type of neural network architecture. The introduction of the Transformer architecture in the paper \"Attention Is All You Need\" in 2017 marked a significant breakthrough in NLP.'}),/*#__PURE__*/t(\"p\",{children:[\"Transformers relied on self-attention mechanisms that allow them to capture long-range dependencies and relationships within the data, leading to improved language understanding. They set foundation models apart from other neural network architectures that may not handle sequences and relationships in the same manner. The transformer architecture laid the foundation for modern \",/*#__PURE__*/e(a,{href:\"https://toloka.ai/large-language-models/\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(n.a,{children:\"large language models\"})}),\" (LLMs).\"]}),/*#__PURE__*/e(\"p\",{children:\"The application of Transformers to modalities other than NLP has been made possible by the development of a Transformer variety called the Vision Transformer (ViT). The ViT model has revolutionized the field of computer vision by demonstrating the efficacy of the Transformer architecture in understanding and processing visual data. ViT's success has inspired the development of various ViT-based models, and it continues to be a subject of active research and innovation in the field of AI and machine learning.\"}),/*#__PURE__*/e(\"h2\",{children:\"Benefits of Foundation Models\"}),/*#__PURE__*/e(\"p\",{children:\"The unique characteristics of foundation models unlock a wide array of benefits that have a profound impact on various AI applications. These benefits are a direct result of their versatility, adaptability, and capabilities in understanding and generating diverse data types.\"}),/*#__PURE__*/t(\"ul\",{style:{\"--framer-font-size\":\"18px\",\"--framer-text-alignment\":\"start\",\"--framer-text-color\":\"rgb(30, 33, 38)\",\"--framer-text-stroke-width\":\"0px\",\"--framer-text-transform\":\"none\"},children:[/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Customization for Diverse Applications\"}),\". The adaptability of foundation models through fine-tuning allows for customization to specific tasks or domains. This flexibility ensures that AI systems are not one-size-fits-all but can be tailored to diverse industries and use cases;\"]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Versatility\"}),\". Foundation models are highly versatile and can be applied to a broad spectrum of tasks due to their multimodality. They can perform tasks related to natural language understanding and generation, image processing, and even multimodal tasks that combine text, image, and audio data;\"]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Efficient Transfer Learning\"}),\". The pre-training and fine-tuning approach in foundation models significantly reduces the time and resources required to develop AI solutions. This efficiency is especially valuable for businesses and developers looking to implement AI in their applications;\"]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Cost and Time Savings\"}),\". Foundation models significantly reduce the time required to develop AI solutions. Instead of building models from scratch, developers can leverage pre-trained models, saving weeks or even months in development time. Once deployed, models based on foundation models often require less ongoing maintenance. The models are designed to be robust and adaptable, reducing the need for continuous adjustments;\"]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Minimal Data Annotation\"}),\". Traditional machine learning models often demand extensive labeled data for training. In contrast, foundation models require fewer labeled examples for fine-tuning, thanks to their knowledge gained during pre-training. This is especially advantageous when labeled data is scarce or costly to obtain.\"]})})]}),/*#__PURE__*/e(\"h2\",{children:\"Applications of Foundation Models\"}),/*#__PURE__*/e(\"h3\",{children:\"Natural Language Processing\"}),/*#__PURE__*/e(\"p\",{children:\"Natural Language Processing applications are one of the most prominent and rapidly evolving domains for foundation models. These models, equipped with advanced language understanding capabilities, are employed in a wide range of NLP applications. Here are some of them:\"}),/*#__PURE__*/t(\"ul\",{style:{\"--framer-font-size\":\"18px\",\"--framer-text-alignment\":\"start\",\"--framer-text-color\":\"rgb(30, 33, 38)\",\"--framer-text-stroke-width\":\"0px\",\"--framer-text-transform\":\"none\"},children:[/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Machine Translation. Foundation Models like BERT have improved the quality of machine translation applications;\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Language Generation. They are employed in chatbots and automated content generation;\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Text Summarization. They can generate concise summaries of long text documents;\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Sentiment Analysis. Foundation models excel in sentiment analysis, determining the emotional tone in text data;\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Speech Recognition. In speech recognition, foundation models convert spoken language into text, serving as the backbone for voice assistants, transcription services, and voice command systems.\"})})]}),/*#__PURE__*/e(\"h3\",{children:\"Computer vision\"}),/*#__PURE__*/e(\"p\",{children:\"Computer vision (CV) is a field of artificial intelligence and computer science that focuses on enabling computers and machines to interpret and understand visual information from the world, similar to how humans perceive and process images and videos. Here are some key CV applications of foundation models:\"}),/*#__PURE__*/t(\"ul\",{style:{\"--framer-font-size\":\"18px\",\"--framer-text-alignment\":\"start\",\"--framer-text-color\":\"rgb(30, 33, 38)\",\"--framer-text-stroke-width\":\"0px\",\"--framer-text-transform\":\"none\"},children:[/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Image Classification. Foundation models like CLIP are capable of classifying images and associating them with their corresponding text descriptions;\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Object Detection. They can identify and locate objects within images;\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Image Captioning. Foundation models generate descriptive captions for images, making it easier to understand the content of pictures;\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Image Generation. Models like DALL-E can create images based on textual prompts;\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Face Recognition. CV foundation models are used in face recognition and facial expression analysis systems;\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Semantic Segmentation. Foundation models can perform semantic segmentation by labeling each pixel in an image with the object or class it belongs to;\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Video Description. Foundation models can be employed for generating text descriptions for video content;\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Visual Question Answering (VQA). VQA systems powered by foundation models can answer questions about the content of images.\"})})]}),/*#__PURE__*/e(\"p\",{children:\"Some of the applications can be considered multimodal because the input and/or output can fall into different categories of modality. Multimodal applications of foundation models leverage their versatility to understand and make connections between different types of data. For example, tasks like image generation from text prompts or image captioning imply the use of visual and textual modality.\"}),/*#__PURE__*/e(\"h2\",{children:\"Examples of Foundation Models\"}),/*#__PURE__*/e(\"h3\",{children:\"Large Language Models (LLMs)\"}),/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(a,{href:\"https://toloka.ai/large-language-models/\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(n.a,{children:\"Large Language Model\"})}),\" is a category of foundation models that has gained significant attention in the field of natural language processing. LLMs are incredibly large in terms of the number of parameters, often ranging from hundreds of millions to tens of billions. This immense scale allows them to capture intricate patterns in language data.\"]}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"strong\",{children:\"GPT (Generative Pre-trained Transformer)\"})}),/*#__PURE__*/e(\"p\",{children:\"Developed by OpenAI, GPT is one of the most famous foundation models. There are 5 basic versions of GPT, each representing a significant leap in scale and performance in comparison to the previous one. GPT-3.5 was used to create the chatbot product known as ChatGPT. It became popular for its ability to engage in conversational interactions and provide informative responses to a variety of user queries. GPT-4, the latest version of GPT, is a multimodal foundation model, capable of text and image processing.\"}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"strong\",{children:\"BERT (Bidirectional Encoder Representations from Transformers)\"})}),/*#__PURE__*/e(\"p\",{children:\"BERT, developed by Google, focuses on bidirectional understanding of text. Models like BERT have a profound impact on our daily lives, often behind the scenes. It's widely used in search engine algorithms to improve the accuracy of search results. BERT analyzes and translates text in machine translation systems like Google Translate, considering context and grammar. This foundation model is a fundamental tool for language understanding in various contexts, including chatbots and virtual assistants. It helps these systems comprehend user queries and generate contextually relevant responses. In applications like Gmail, BERT can predict and suggest text as users compose emails, making text input more efficient and user-friendly. While many foundation models are primarily associated with NLP and understanding, as was mentioned earlier some models have been developed to work with images and other types of data. These multimodal foundation models are also typically based on deep learning architectures, similar to their text-only processing counterparts.\"}),/*#__PURE__*/e(\"h3\",{children:\"Vision-Language Foundation Models\"}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"strong\",{children:\"CLIP (Contrastive Language-Image Pretraining)\"})}),/*#__PURE__*/e(\"p\",{children:\"Foundation models like CLIP, introduced by OpenAI in 2021, extended the Transformer architecture to understand and generate text and images. This opened up new possibilities for AI in CV, multimodal applications, and cross-modal understanding. CLIP, like other foundation models in a vision-language domain, is pre-trained on a large corpus of text and image data, which enables it to learn the relationships between visual and textual content.\"}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"strong\",{children:\"DALL\\xb7E\"})}),/*#__PURE__*/e(\"p\",{children:\"Also developed by OpenAI, DALL\\xb7E is a model capable of generating images from textual descriptions. It is a multimodal implementation of GPT. It takes text prompts and produces corresponding images. It can produce highly creative and contextually relevant images based on textual descriptions.\"}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"strong\",{children:\"Stable Diffusion\"})}),/*#__PURE__*/e(\"p\",{children:\"Stable Diffusion is a foundation model with generative AI capabilities. This text-to-image tool employs a kind of deep generative AI neural network called the latent diffusion model. Users can instruct the model to create diverse high-quality images. The model allows the use of prompts to partially alter existing images via inpainting (adding new elements to an image) and outpainting (removing elements from an image).\"}),/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"GPT-4\"}),' One notable advancement in GPT-4, distinct from its predecessors, is its potential to handle image inputs. It\\'s important to note that this feature is currently labeled as a \"research preview\" and is not yet accessible to the general public. GPT-4 impressively demonstrates its proficiency in accurately understanding intricate visuals, including charts, memes, and scientific papers.']}),/*#__PURE__*/e(\"h3\",{children:\"Vision-Audio-Language Foundation Models\"}),/*#__PURE__*/e(\"p\",{children:\"Vision-Audio-Language (VAL) foundation models are a subclass of multimodal models that have the ability to understand and work with data from three primary modalities: images, audio, and text. These models are designed to process, generate, and make connections between data in these modalities\"}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"strong\",{children:\"VALOR (Vision-Audio-Language Omni-Perception Pretraining Model)\"})}),/*#__PURE__*/e(\"p\",{children:\"VALOR is designed to understand and create information from multiple types of data, such as pictures (vision), sounds (audio), and words (language). It's different from previous models that mainly focus on connecting pictures and words. VALOR combines all three types of data to get a better understanding.\"}),/*#__PURE__*/e(\"p\",{children:\"VALOR is made up of three separate parts that work with each type of data. These parts are called encoders. There is also a part called a decoder that can put all this information together and create text based on what the model sees, hears, or both.\"}),/*#__PURE__*/e(\"h2\",{children:\"Challenges of Foundation Models\"}),/*#__PURE__*/e(\"p\",{children:\"Foundation models, while powerful and versatile, come with several challenges that need to be addressed for their effective and responsible use.\"}),/*#__PURE__*/e(\"h3\",{children:\"Fake Content\"}),/*#__PURE__*/e(\"p\",{children:\"Misinformation is a significant challenge associated with the use of foundation models, particularly those designed for natural language generation. Foundation models can inadvertently amplify misinformation by generating text that sounds authoritative and convincing. These models also enable the creation of deepfake content, including manipulated videos, audio, and images. Deepfakes can be used to impersonate public figures or create false evidence, leading to misinformation and distrust. Unfortunately, traditional methods for detecting misinformation are less effective against content generated by advanced foundation models, as they can closely mimic humans.\"}),/*#__PURE__*/e(\"h3\",{children:\"Legal and ethical considerations\"}),/*#__PURE__*/e(\"p\",{children:\"Generating content using foundation models can raise questions about intellectual property rights. If a model generates text, art, or music, who owns the resulting content, and are there copyright implications? Ownership and control over the content generated by foundation models are unclear. This becomes a legal issue when content is used for commercial purposes or when disputes arise over authorship.\"}),/*#__PURE__*/e(\"p\",{children:\"Discriminatory or biased content generated by foundation models can lead to legal challenges, especially in the context of anti-discrimination laws. When foundation models are used to generate harmful, illegal, or fraudulent content, legal issues arise. The responsibility for preventing misuse and taking legal action against malicious users can be complex.\"}),/*#__PURE__*/e(\"h3\",{children:\"Data Privacy\"}),/*#__PURE__*/e(\"p\",{children:\"The massive scale of foundation models raises concerns about data privacy. Pretraining on large datasets means that sensitive information may be present in the model's parameters, potentially posing privacy risks. Obtaining informed consent from individuals for collecting and using their data can be challenging. Users may not fully understand the extent to which their data will be used, making it difficult to obtain meaningful consent.\"}),/*#__PURE__*/e(\"h2\",{children:\"Conclusion\"}),/*#__PURE__*/e(\"p\",{children:\"Foundation models are an outstanding breakthrough in the development of artificial intelligence, significantly transforming the landscape of natural language processing, computer vision, and multimodal tasks. These models, often based on the powerful Transformer architecture, have the capacity to understand, generate, and interpret a wide range of data.\"}),/*#__PURE__*/e(\"p\",{children:\"Their applications are vast, and their benefits are substantial, though they come with their share of challenges. As these models continue to evolve, it's crucial to use them responsibly and ethically to utilize their full potential for the betterment of society. Troubleshooting challenges associated with disinformation and fake content, ethical considerations, data privacy concerns, and regulatory compliance remains a critical issue. Accountability for the development and implementation of these models is vital to ensure that their impact on society is positive and fair. Foundation models have given us the ability to interact with machines more naturally, creating new possibilities for content generation, and pushing the boundaries of human knowledge. They serve as a reminder of the infinite range of opportunities that AI brings to the table.\"})]});export const richText3=/*#__PURE__*/t(i.Fragment,{children:[/*#__PURE__*/e(\"p\",{children:\"Generative artificial intelligence emerged quite recently and has already revolutionized various business industries. It is in a very early stage of its development, but it's already not just a buzzword; it's a powerful tool that is redefining how businesses operate, strategize, and engage with customers.\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI has already demonstrated its potential to enhance efficiency and creativity across multiple domains. It can create images and photos, synthesize music and voice, but an incredible breakthrough happened when generative AI tools gained the ability to comprehend and generate cohesive and human-like texts.\"}),/*#__PURE__*/e(\"p\",{children:\"Its unparalleled ability to create textual content with context, coherence, and creativity has made it indispensable in modern business operations. As we explore generative AI applications, we'll focus on how it helps businesses stay competitive, agile, and ready for the future, particularly as it pertains to text generation.\"}),/*#__PURE__*/e(\"h2\",{children:\"Generative AI in a Nutshell\"}),/*#__PURE__*/e(\"p\",{children:\"Generative Artificial Intelligence, as the name suggests, generates content. It is opposed to traditional AI, which does not create any new information but accomplishes important tasks of systematization, recognition, and prediction.\"}),/*#__PURE__*/e(\"p\",{children:\"The main difference between the two types of AI is that the former uses generative models to accomplish its job, while the latter employs discriminative models. Discriminative Models focus on learning the boundary that separates different classes or categories in data. Generative models on the other hand aim to learn the underlying probability distribution of the data. They can generate new data samples that are similar to the training data.\"}),/*#__PURE__*/e(\"p\",{children:\"Another pivotal part of the generative AI ecosystem is foundation models. Generative AI utilizes their power to generate content, whether it's generating creative stories, composing music, or realistic photos or human-like responses in conversational agents.\"}),/*#__PURE__*/e(\"p\",{children:\"The adaptability of generative AI foundation models allows them to be fine-tuned for specific generative tasks. They serve as a base for other applications. Apps built on top of them may have absolutely zero in common, yet have the same model at the core. The vast knowledge and linguistic capabilities embedded in such models enable generative AI to produce coherent and contextually relevant content, making it a valuable tool for various applications.\"}),/*#__PURE__*/e(\"p\",{children:\"Multimodal foundation models are designed to process and understand data from multiple modalities, which can include not only text but also images, audio, and more. These models have the capacity to bridge the gap between different types of data and enable comprehensive analysis and generation across various media types.\"}),/*#__PURE__*/e(\"h2\",{children:\"Examples of Generative AI models\"}),/*#__PURE__*/t(\"ul\",{style:{\"--framer-font-size\":\"18px\",\"--framer-text-alignment\":\"start\",\"--framer-text-color\":\"rgb(30, 33, 38)\",\"--framer-text-stroke-width\":\"0px\",\"--framer-text-transform\":\"none\"},children:[/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"GPT-4 (Generative Pre-trained Transformer 4)\"}),\". Developed by OpenAI, GPT-4 is one of the largest multimodal models with 1.76 trillion parameters. This latest version of GPT is capable of processing requests in the form of images and text, and then producing new text responses;\"]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"BERT (Bidirectional Encoder Representations from Transformers)\"}),\". Google's BERT is a large language model that revolutionized natural language understanding by pre-training on vast amounts of text data. It's used for various NLP tasks, including sentiment analysis, question-answering, and text classification;\"]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"PaLM-E (Pathways Language Model - Embodied)\"}),\". PaLM-E is a multimodal language model for controlling robots. It is a combination of Google's PaLM and Vision Transformer (ViT) models. Input data for the PaLM-E model are multimodal sentences that combine text and visual embeddings, which allows the model to successfully solve various tasks like planning sequential manipulations, visual responses to questions, etc.\"]})})]}),/*#__PURE__*/e(\"h2\",{children:\"Generative AI Use Cases\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI capabilities encompass a broad spectrum of advanced functions that employ the power of artificial intelligence to create, generate, and enhance content across various domains. Here are some of its key capabilities and use cases:\"}),/*#__PURE__*/e(\"h3\",{children:\"Text generation\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI can craft human-like text, making it invaluable for content generation, conversational AI tools like chatbots, and creative writing. The natural language capabilities of generative AI allow us to create content that is not only contextually accurate but also creatively compelling. Natural language processing (NLP), a subset of AI, provides machines with the ability to understand and manipulate human language. In the context of generative AI, this entails text generation that possesses an astonishingly human-like fluency and coherence.\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI can help businesses extract valuable insights from large volumes of text data, translate content from one language to another quickly and accurately, streamline the process of preparing emails, presentations, and documents and more.\"}),/*#__PURE__*/e(\"h3\",{children:\"Music Generation\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI can compose original melodies, harmonies, and musical compositions. It's not limited to a specific genre and can generate music in classical, jazz, pop, or any style you desire. Musicians and producers can collaborate with generative AI tools to expedite music production. Generative AI can come up with an almost endless stream of musical ideas, making it a valuable tool for composers seeking inspiration or musicians looking to collaborate with an AI co-creator.\"}),/*#__PURE__*/e(\"h3\",{children:\"Realistic speech audio generation\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI powers Text-to-Speech (TTS) systems that convert written text into spoken words. These systems produce speech that closely mimics human vocal characteristics, including tone, pitch, and rhythm. Advanced TTS models have a nuanced understanding of language, allowing them to generate speech with natural pronunciation and intonation. Generative AI technology enables voice customization, where users can modify speech characteristics, such as gender, accent, and age, to align with specific needs or brand identities.\"}),/*#__PURE__*/e(\"h3\",{children:\"Code generation\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI can automate repetitive tasks, such as writing boilerplate code, generating database queries, or formatting code according to specific coding standards. Integrated code generation tools can provide real-time code suggestions and auto-completion as developers write code. This enhances productivity and streamlines software development. Generative AI can also identify and fix code defects or suggest improvements for code optimization.\"}),/*#__PURE__*/e(\"h3\",{children:\"Computer Vision and Image Generation\"}),/*#__PURE__*/e(\"p\",{children:\"Computer vision is a field of AI that focuses on enabling computers to interpret and understand visual information from the world, much like the human visual system. It involves the development of algorithms and models to enable computers to analyze, process, and make decisions based on visual data from images and videos.\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI models can produce realistic images and visually striking artworks, paintings, and illustrations. All this owing to the discovery and development of algorithms such as Generative Adversarial Networks (GANs) and a subcategory of deep generative models called Diffusion Models.\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI assists in generating visuals for marketing campaigns, advertisements, and promotional materials. It can create eye-catching images and graphics tailored to specific brand messages and target audiences.\"}),/*#__PURE__*/e(\"p\",{children:\"In computer vision tasks like object detection and image classification, generative AI can create synthetic images. These images are used to expand training datasets, improving the performance and robustness of machine learning models. Generative AI generates realistic content for virtual and augmented reality environments. It creates 3D models, textures, and simulations that make virtual experiences more immersive.\"}),/*#__PURE__*/e(\"p\",{children:\"In e-commerce and image-based search engines, computer vision identifies products or objects within images. Generative AI enhances recommendation systems by providing more accurate and relevant suggestions based on such visual content.\"}),/*#__PURE__*/e(\"h2\",{children:\"Why does Text Generation Play a Key Part in Generative AI Development?\"}),/*#__PURE__*/e(\"p\",{children:\"All cases of human-like text usage and processing by a computer are called Natural Language Processing (NLP). In the 1980s, machine learning was introduced for computer language processing.\"}),/*#__PURE__*/e(\"p\",{children:\"By the 2020s generative AI has advanced rapidly, as the capacity and computational power of devices has increased due to new technological developments. The field of NLP has received a significant boost. Generative AI tools capable of generating texts that are indistinguishable from those written by humans have appeared. They can now perform very high-quality translations from one language to another, extract data from the text presented to them, create document templates, etc. So why is generative AI especially focused on text processing?\"}),/*#__PURE__*/e(\"p\",{children:\"The answer may lie in the fact that text is the basis of perhaps all human thought processes. Everything begins with text and language: a painting first appears as a thought in the artist's head, melodies are also recorded using the language of notes, and even our inner voice is also a text. This is probably why one of the first breakthroughs in the development of generative AI was the ability of AI tools to generate and process text in every possible way.\"}),/*#__PURE__*/e(\"p\",{children:\"Today, in order to generate a picture in a generative AI application, we have to enter a prompt containing a textual message that tells the algorithm what we want to see in the picture. Generative AI also generates music and sounds based on text requests. This is quite similar to how a person's thought process works.\"}),/*#__PURE__*/e(\"p\",{children:\"Artificial intelligence aims to learn to think and create like a human. Therefore, figuratively speaking, in order to achieve this goal in the future, data scientists have decided to start (or should we rather say continue) the development of generative AI by mastering its use of text, as humans also start any kind of interaction with the world around them using text.\"}),/*#__PURE__*/e(\"p\",{children:\"Almost all modern generative AI tools can work with text, but not all can work with visual data. Although multimodal systems capable of processing both text and image are gradually gaining momentum.\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI can accomplish certain tasks better and faster than a human being. It may become a competitive advantage for many businesses. Nowadays, generative AI is becoming an integral part of both large and small businesses.\"}),/*#__PURE__*/e(\"h2\",{children:\"Business Applications of Text Generation\"}),/*#__PURE__*/e(\"p\",{children:\"By integrating text generation with generative AI into various aspects of business operations, organizations can improve productivity, reduce costs, provide better customer experiences, and gain a competitive edge in today's data-driven and digitally connected world. That's how organizations can employ generative AI in their processes:\"}),/*#__PURE__*/e(\"h3\",{children:\"Generating Documentation\"}),/*#__PURE__*/e(\"p\",{children:\"Generating documentation through generative AI tools can streamline the process of creating manuals, guides, reports, and other types of documents. Generative AI reduces manual effort, minimizes errors, and ensures that critical information is always available to employees, customers, and stakeholders.\"}),/*#__PURE__*/e(\"p\",{children:\"Leveraging generative AI document generation can provide a competitive edge by improving efficiency and the ability to adapt to changing market dynamics. These are some insights concerning documentation generation with the help of generative AI:\"}),/*#__PURE__*/t(\"ul\",{style:{\"--framer-font-size\":\"18px\",\"--framer-text-alignment\":\"start\",\"--framer-text-color\":\"rgb(30, 33, 38)\",\"--framer-text-stroke-width\":\"0px\",\"--framer-text-transform\":\"none\"},children:[/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Product descriptions\"}),\". E-commerce businesses can employ AI to generate product descriptions and catalogs. This is particularly useful for maintaining large product inventories and ensuring consistency in product offerings;\"]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Reports and presentations\"}),\". Generative AI assists in generating reports, market analyses, financial summaries, and business plans. It can summarize complex data into concise, readable reports for stakeholders and decision-makers;\"]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Email sequences\"}),\". AI-powered systems can assist employees in composing professional emails, offering suggestions for subject lines, body content, and titles;\"]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Legal and finance\"}),\". These functions benefit from generative AI in drafting contracts, agreements, and reports;\"]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Documents versioning\"}),\". AI can automatically update documents in real time based on changing data or variables, ensuring that documents are always current and accurate.\"]})})]}),/*#__PURE__*/e(\"h3\",{children:\"Translating Documents\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI can translate documents and text from one language to another accurately and quickly, facilitating global communication and expanding market reach, allowing businesses to reach a broader audience. Traditional human translation can be time-consuming, especially for lengthy documents. Generative AI can translate documents rapidly, saving valuable time and resources.\"}),/*#__PURE__*/e(\"p\",{children:\"AI-powered document translation allows businesses to tap into new markets while maintaining language accuracy and consistency. Some use cases of translating documents with the help of generative AI for business include:\"}),/*#__PURE__*/t(\"ul\",{style:{\"--framer-font-size\":\"18px\",\"--framer-text-alignment\":\"start\",\"--framer-text-color\":\"rgb(30, 33, 38)\",\"--framer-text-stroke-width\":\"0px\",\"--framer-text-transform\":\"none\"},children:[/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Translating financial reports, investment summaries, and market analysis documents allows businesses to communicate financial insights to international stakeholders effectively;\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Businesses can translate marketing materials, advertisements, product descriptions, and website content to reach a wider global audience. AI ensures that the translated content aligns with the brand's messaging and style.\"})})]}),/*#__PURE__*/e(\"h3\",{children:\"Marketing Strategies and Content Creation\"}),/*#__PURE__*/e(\"p\",{children:\"AI-powered tools generate large volumes of marketing content quickly, including landing pages, blog and social media posts, and ad copies, helping businesses maintain a strong online presence and engage with their target audience effectively. Here are some features of generative AI regarding generating textual content for marketing purposes:\"}),/*#__PURE__*/t(\"ul\",{style:{\"--framer-font-size\":\"18px\",\"--framer-text-alignment\":\"start\",\"--framer-text-color\":\"rgb(30, 33, 38)\",\"--framer-text-stroke-width\":\"0px\",\"--framer-text-transform\":\"none\"},children:[/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Generative AI can produce a wide range of content at a scale that would be challenging for human writers alone. This enables businesses to consistently update their website and engage with their audience through content marketing;\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"AI-powered brainstorming sessions can assist marketing teams in generating a wide range of ideas quickly. AI can suggest various angles, themes, and approaches for marketing campaigns;\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Generative AI can assist in creating content plans by generating ideas and organizing them into a structured timeline for consistent and strategic content delivery.\"})})]}),/*#__PURE__*/e(\"h3\",{children:\"Autonomous Agents and Chatbots\"}),/*#__PURE__*/e(\"p\",{children:\"Chatbots and virtual assistants utilize generative AI to engage users in natural conversations. They provide customer support, answer queries, and offer personalized recommendations, enhancing customer service. Incorporating autonomous agents and chatbots into business processes enhances efficiency, scalability, and customer satisfaction. Such AI-powered agents can handle routine tasks, provide instant responses, and generate content that resonates with customers, ultimately driving business growth and competitiveness. Their features include:\"}),/*#__PURE__*/t(\"ul\",{style:{\"--framer-font-size\":\"18px\",\"--framer-text-alignment\":\"start\",\"--framer-text-color\":\"rgb(30, 33, 38)\",\"--framer-text-stroke-width\":\"0px\",\"--framer-text-transform\":\"none\"},children:[/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Autonomous agents and chatbots can provide 24/7 customer support, ensuring that customers receive assistance even outside business hours;\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Chatbots can answer common customer queries by generating responses based on a pre-defined FAQ database, reducing the workload on customer support teams;\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Virtual assistants can troubleshoot and resolve customer issues by providing step-by-step instructions or generating solutions.\"})})]}),/*#__PURE__*/e(\"h3\",{children:\"Personalized Product Recommendations\"}),/*#__PURE__*/e(\"p\",{children:\"E-commerce platforms use AI-generated product recommendations based on user browsing and purchasing history, which allows them to boost customer satisfaction. Personalized product recommendations powered by generative AI models offer businesses a powerful tool to enhance customer engagement, increase sales, and improve user experience. Customers are more likely to return to a business that consistently recommends products they find relevant and valuable, stimulating long-term loyalty.\"}),/*#__PURE__*/e(\"p\",{children:\"Therefore personalized recommendations created with the help of generative AI can significantly contribute to increasing the average purchase value and click-through rates (CTR) in an online store as customers are more likely to engage with and act on personalized recommendations that align with their interests and needs. Here's what AI can do in terms of personalized recommendations:\"}),/*#__PURE__*/t(\"ul\",{style:{\"--framer-font-size\":\"18px\",\"--framer-text-alignment\":\"start\",\"--framer-text-color\":\"rgb(30, 33, 38)\",\"--framer-text-stroke-width\":\"0px\",\"--framer-text-transform\":\"none\"},children:[/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Generative AI creates user profiles by analyzing the data. These profiles capture individual preferences, interests, and behaviors. The AI identifies patterns and trends in the data to understand what each user is likely to be interested in;\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"AI can divide customers into groups based on behavior, demographics, and preferences, allowing businesses to provide targeted recommendations to each such group;\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Generative AI can provide real-time recommendations as users browse an online store or platform. This guarantees that customers are presented with the most relevant products based on their current interests and browsing behavior.\"})})]}),/*#__PURE__*/e(\"h3\",{children:\"Market Research and Sentiment Analysis\"}),/*#__PURE__*/e(\"p\",{children:\"AI-driven text analysis tools can process and summarize vast amounts of textual data, extracting valuable insights about market trends, customer sentiment, and competitor strategies. AI can identify emerging trends, consumer preferences, and market shifts by analyzing textual data. This information helps businesses stay ahead of the competition and adapt their strategies accordingly. The following generative AI use cases aid in conducting effective market research and sentiment analysis:\"}),/*#__PURE__*/t(\"ul\",{style:{\"--framer-font-size\":\"18px\",\"--framer-text-alignment\":\"start\",\"--framer-text-color\":\"rgb(30, 33, 38)\",\"--framer-text-stroke-width\":\"0px\",\"--framer-text-transform\":\"none\"},children:[/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Generative AI can analyze customer feedback, reviews, and social media conversations to determine sentiment. It identifies positive, negative, or neutral sentiments and helps businesses understand how customers perceive their products or services;\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Sentiment analysis carried out by generative AI can pinpoint areas where products or services excel and areas that need improvement. This information is invaluable for refining offerings to meet customer expectations;\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Businesses can use generative AI text processing tools to monitor competitors' activities and sentiments expressed about them. This data aids businesses in refining their competitive strategies.\"})})]}),/*#__PURE__*/e(\"h3\",{children:\"Content Moderation\"}),/*#__PURE__*/e(\"p\",{children:\"Content moderation with generative AI is essential for businesses to maintain a safe and respectful online environment, comply with regulations, and protect their brand reputation. Businesses often have social media pages for engagement and marketing. AI text processing assists in automating content moderation on such platforms.\"}),/*#__PURE__*/e(\"p\",{children:\"It can detect and filter out inappropriate or harmful content, offensive, or spam comments, ensuring a positive online presence and appropriate social media posts. By preventing the spread of harmful or offensive content associated with the brand, content moderation safeguards brand reputation and customer trust. Some prominent generative AI use cases for content moderation include:\"}),/*#__PURE__*/t(\"ul\",{style:{\"--framer-font-size\":\"18px\",\"--framer-text-alignment\":\"start\",\"--framer-text-color\":\"rgb(30, 33, 38)\",\"--framer-text-stroke-width\":\"0px\",\"--framer-text-transform\":\"none\"},children:[/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"E-commerce businesses can use content moderation to ensure that customer reviews are genuine and constructive while filtering out fake reviews or hate speech;\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"AI can assist in filtering out spam, phishing emails, and other malicious content from business email inboxes, enhancing email security;\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Video platforms and live streaming services can use AI to detect and remove inappropriate comments, hate speech, and harmful content from chat sections during live broadcasts\"})})]}),/*#__PURE__*/e(\"h2\",{children:\"Advantages of Generative AI Tools in Business\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI offers numerous advantages to businesses and various industries. These tools, powered by natural language processing, have the potential to revolutionize operations, enhance customer experiences, and drive growth in organizations. Here are some key advantages of using generative AI tools in business:\"}),/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Scalability\"}),\". Generative AI can quickly generate content at scale, allowing businesses to handle increased workloads and adapt to changing demands. Businesses can also easily scale content generation and customer support efforts without proportionally increasing human labor. AI can handle increased workloads effortlessly;\"]}),/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Cost Efficiency\"}),\". Generative AI can automate content generation, reducing the need for manual content creation, writing, and editing. This leads to cost savings in content production and marketing efforts;\"]}),/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Time Savings\"}),\". AI-driven automation accelerates various tasks, such as drafting documents, generating reports, and responding to customer inquiries, freeing up employees' time for more strategic and value-added activities;\"]}),/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Improved Efficiency\"}),\". Generative AI automates time-consuming and repetitive tasks, allowing employees to focus on higher-value activities and strategic initiatives;\"]}),/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Competitive Advantage\"}),\". Businesses adopting generative AI techniques can gain a competitive advantage by being able to stay on top of technological advancements and offer a superior customer experience due to its use.\"]}),/*#__PURE__*/e(\"h2\",{children:\"Standout Generative AI Use Cases\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI solutions have proven to be transformative tools for big companies, as they continue to discover unique and unconventional applications across various industries.\"}),/*#__PURE__*/e(\"h3\",{children:\"E-commerce\"}),/*#__PURE__*/e(\"p\",{children:'The e-commerce megacorporation Amazon has introduced a feature in its shopping app that employs AI to condense customer reviews into concise summaries. These summaries offer a quick overview of what customers appreciated and disliked about a product, accompanied by a disclaimer indicating that the summary has been \"AI-generated from the text of customer reviews.\"'}),/*#__PURE__*/e(\"p\",{children:\"The use of AI to summarize reviews is a valuable tool for online shoppers, considering Amazon's vast catalog of products, each potentially having thousands of reviews. It simplifies the decision-making process for customers. It's also worth mentioning that Amazon has a long history of using AI and machine learning to provide its customers with personalized recommendations and targeted advertising.\"}),/*#__PURE__*/e(\"h3\",{children:\"Healthcare\"}),/*#__PURE__*/e(\"p\",{children:\"Pharmaceutical giants like Pfizer employ a generative AI model to manufacture medicines that are more likely to perform well. The company can assess a vast amount of clinical data to uncover deeper insights and enhance the efficiency of the discovered drugs. That way generative AI accelerates the process of drug selection process and Pfizer identifies potential drug candidates with a higher probability of success.\"}),/*#__PURE__*/e(\"p\",{children:\"Pfizer also employs generative AI models to identify and screen for rare diseases, such as transthyretin amyloid cardiomyopathy (ATTR-CM). The implementation of these technologies allows medical professionals and patients to detect the symptoms of rare diseases more accurately and at an earlier stage.\"}),/*#__PURE__*/e(\"h3\",{children:\"Film industry\"}),/*#__PURE__*/e(\"p\",{children:\"In content creation, film studios like Warner Bros. have embraced generative AI to forecast the box office of their movies. Machine learning algorithms analyze a wide array of data sources, including past box office statistics, rentals, and even pirated downloads. Taking advantage of vast data sets and insights from the generative AI models, the AI platform employed by Warner is set to free studio managers from laborious, iterative duties, empowering them to instead concentrate on delivering practical insights to drive key decisions regarding product packaging, distribution, marketing, and sales.\"}),/*#__PURE__*/e(\"p\",{children:\"This transition toward data-driven decision-making has the potential to revolutionize the movie industry, offering not only a deeper understanding of audience preferences but also the ability to make more strategic and profitable choices in film development.\"}),/*#__PURE__*/e(\"h3\",{children:\"Marketing\"}),/*#__PURE__*/e(\"p\",{children:\"Coca-Cola, a global beverage corporation, is adopting generative AI for innovative advertising campaigns. The company has taken significant steps to incorporate AI into its marketing strategies. For instance, the company created an advertisement set in a museum where famous artworks were brought to life through the use of OpenAI's DALL-E 2 generative image tool. This innovative approach not only added an element of interactivity to the ad but also showcased the creative potential of generative AI.\"}),/*#__PURE__*/e(\"p\",{children:\"Coca-Cola's proactive approach in integrating generative AI into its advertising strategies not only demonstrates its forward-thinking nature but also serves as an example of how AI can be a powerful tool in the hands of creative professionals.\"}),/*#__PURE__*/e(\"p\",{children:\"These are just a few examples of how enterprises are utilizing generative AI. As companies increasingly recognize the potential of these technologies, we can anticipate even more innovative applications in the future, further transforming the business landscape and opening up new possibilities for efficiency, creativity, and customer engagement.\"}),/*#__PURE__*/e(\"h2\",{children:\"Conclusion\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI has proven to be transformative in the world of business, especially in tasks related to text generation. Its applications extend to content creation, language translation, market research, personalized recommendations, content moderation, and beyond.\"}),/*#__PURE__*/e(\"p\",{children:\"The field of generative AI is dynamic, with ongoing advancements and new models emerging every year. To stay competitive businesses must embrace continuous learning and adaptation to leverage the latest technologies effectively.\"}),/*#__PURE__*/e(\"p\",{children:\"Incorporating generative AI into business operations offers a competitive edge, enhances customer interactions, and streamlines various business functions. As generative AI advances, companies that strategically leverage this technology will be better positioned to succeed in a world that is becoming more and more dominated by data and automation.\"})]});export const richText4=/*#__PURE__*/t(i.Fragment,{children:[/*#__PURE__*/e(\"p\",{children:\"Large language models have wowed us all with their ability to invent a story, write a poem, or craft an email. But the impact of a model with creative writing capabilities reaches far beyond the wow factor. Businesses anticipate large gains in employee productivity and accelerated growth by leveraging generative AI for writing tasks. \"}),/*#__PURE__*/e(\"p\",{children:\"The question is: how do language models develop creative writing skills?\"}),/*#__PURE__*/e(\"p\",{children:\"The secret lies in designing interesting prompts for model training. Generative models rely on prompts, which means that the quality of training prompts can shape the quality of the model\u2019s output.\"}),/*#__PURE__*/t(\"p\",{children:[\"In a \",/*#__PURE__*/e(a,{href:\"https://toloka.ai/blog/tech-spec-for-your-own-llm/\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(n.a,{children:\"recent post\"})}),\" on technical specifications for building your own LLM, we shared the exciting news that Toloka is working on a joint project to build a large language model from scratch. In today\u2019s post, we\u2019ll share some first-hand details on how we collect the best prompts to help the model develop essential creative writing skills.\"]}),/*#__PURE__*/e(\"h2\",{children:\"Aren\u2019t LLMs already good at writing?\"}),/*#__PURE__*/e(\"p\",{children:\"Large language models are innately good at generating texts. They are built to predict the next word and combine words into coherent narratives. But they need to learn how to follow the prompt and answer in a particular style \u2014 that\u2019s where fine-tuning comes in.\"}),/*#__PURE__*/e(\"p\",{children:\"To make LLMs better at creative writing, we need strong prompts that can elicit unique and interesting texts for fine-tuning the model. Here we run into two limitations: a lack of diverse datasets for training, and a lack of benchmarks for evaluating quality.\"}),/*#__PURE__*/e(\"h2\",{children:\"The dataset problem: synthetic vs organic data\"}),/*#__PURE__*/t(\"p\",{children:[\"There are many open datasets available for training, consisting of sets of prompts and good responses (these include \",/*#__PURE__*/e(a,{href:\"https://huggingface.co/datasets/RyokoAI/ShareGPT52K\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(n.a,{children:\"ShareGPT\"})}),\", WizardLM, OpenAssistant, Dolly, and Alpaca). However, most of these datasets are synthetic. In other words, the prompts and answers don\u2019t always express ideas that people are interested in. Some training datasets even use prompts generated by the models themselves.\"]}),/*#__PURE__*/e(\"p\",{children:\"The goal of our project is to help the language model develop authentic human-sounding writing skills, so we\u2019re generating an organic set of training data written fully by humans. While ours is certainly not the first LLM to use human-written prompt sets, we embarked on the project with a mission to make the scope and diversity of the data unique. We take full advantage of Toloka\u2019s global crowd to collect the most diverse prompts imaginable.\"}),/*#__PURE__*/e(\"h2\",{children:\"What makes a good prompt\"}),/*#__PURE__*/e(\"p\",{children:\"A good prompt is organic, and we invite Tolokers from all over the world to contribute prompts that draw on their own personal experience. They are assigned prompt writing tasks in different categories (essay, story, blog post, email, tweet, ad, movie script, and others) and they are free to write a prompt on any topic that interests them \u2014 as long as they follow our tips for the category in the task. Tolokers craft a prompt and submit it, without writing an answer for it. The text responses are later put together by experts in a separate project, and we\u2019ll share those details in an upcoming post.\"}),/*#__PURE__*/e(\"p\",{children:\"Our instructions encourage Tolokers to be creative and come up with prompts that contain plenty of context and aren\u2019t too trivial or generic (\u201CWrite a poem about love\u201D is a bad prompt). The guidelines for the task include examples of interesting prompts and not-so-interesting prompts. For instance, \u201CWrite a short story about a dog\u201D is not interesting enough to evoke a creative response, but \u201CWrite a story about a timid dog who traveled to outer space to save the planet\u201D provides a springboard for creativity.\"}),/*#__PURE__*/e(\"p\",{children:\"Here is an excerpt from the instructions and a submitted prompt.\"}),/*#__PURE__*/e(\"img\",{alt:\"Example of submitted prompt\",className:\"framer-image\",height:\"523\",src:\"https://framerusercontent.com/images/0Vt2ZOYi5cDVi68FTbN2jRwbhU.jpeg\",srcSet:\"https://framerusercontent.com/images/0Vt2ZOYi5cDVi68FTbN2jRwbhU.jpeg?scale-down-to=512 512w,https://framerusercontent.com/images/0Vt2ZOYi5cDVi68FTbN2jRwbhU.jpeg?scale-down-to=1024 1024w,https://framerusercontent.com/images/0Vt2ZOYi5cDVi68FTbN2jRwbhU.jpeg 1881w\",style:{aspectRatio:\"1881 / 1046\"},width:\"940\"}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"Example of submitted prompt\"})}),/*#__PURE__*/e(\"h2\",{children:\"Creative writing prompts spotted in the wild\"}),/*#__PURE__*/e(\"p\",{children:\"The best prompts include full context, like the topic, the basic plot and characters, or even an outline with the desired structure. To help Tolokers with writer\u2019s block, we recommend making it personal \u2014 maybe there\u2019s a topic that\u2019s been on their mind lately, their child asked for a story, or they need to write something for work. Anything that is relevant to them or someone they know is a great subject for a prompt.\"}),/*#__PURE__*/e(\"p\",{children:\"Here are some examples of writing prompts recently submitted by Tolokers.\"}),/*#__PURE__*/e(\"img\",{alt:\"Examples of good prompts\",className:\"framer-image\",height:\"729\",src:\"https://framerusercontent.com/images/A4SDgs2RhofqL4ZJyveZYtmQKiY.jpeg\",srcSet:\"https://framerusercontent.com/images/A4SDgs2RhofqL4ZJyveZYtmQKiY.jpeg?scale-down-to=512 512w,https://framerusercontent.com/images/A4SDgs2RhofqL4ZJyveZYtmQKiY.jpeg?scale-down-to=1024 1024w,https://framerusercontent.com/images/A4SDgs2RhofqL4ZJyveZYtmQKiY.jpeg 1920w\",style:{aspectRatio:\"1920 / 1458\"},width:\"960\"}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"Examples of good prompts\"})}),/*#__PURE__*/e(\"h2\",{children:\"The evaluation problem: how to verify prompt quality\"}),/*#__PURE__*/e(\"p\",{children:\"Now let\u2019s consider the second roadblock: checking prompt quality. Benchmarks that evaluate the quality of models generally compare a text answer to the \u201Cgolden\u201D answer, word for word. But this isn\u2019t a meaningful way to measure the quality of creative texts, which should be unique by definition.\"}),/*#__PURE__*/e(\"p\",{children:\"The only reasonable way to judge creativity is to collect the opinions of real people. Unfortunately, it\u2019s prohibitively expensive and time-consuming to read through thousands of texts. We\u2019ve implemented a better solution: an automated human verification pipeline using the Toloka crowd.\"}),/*#__PURE__*/e(\"p\",{children:\"Tolokers who perform verification tasks are given a prompt written by another person, along with the guidelines. They decide if the prompt follows the guidelines or not \u2014 a simple question of \u201Cgood\u201D or \u201Cbad\u201D. The key here is to make the guidelines as clear and specific as possible, with plenty of examples of what makes a prompt good or bad.\"}),/*#__PURE__*/e(\"img\",{alt:\"Example of a verification task\",className:\"framer-image\",height:\"857\",src:\"https://framerusercontent.com/images/zNHMarfPkzBe2bOXrTxHcVUQTaw.jpeg\",srcSet:\"https://framerusercontent.com/images/zNHMarfPkzBe2bOXrTxHcVUQTaw.jpeg?scale-down-to=1024 937w,https://framerusercontent.com/images/zNHMarfPkzBe2bOXrTxHcVUQTaw.jpeg 1569w\",style:{aspectRatio:\"1569 / 1714\"},width:\"784\"}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"Example of a verification task\"})}),/*#__PURE__*/e(\"p\",{children:\"Since this works like a binary text classification task, we can use our proven quality control methods in the pipeline to make sure verification is accurate. One of our core methods is overlap, meaning we assign multiple people to verify the same prompt and then aggregate their answers to get a reliable final verdict. We use the prompts that \u201Cpass\u201D for model training and discard the ones that are rejected. With our large crowd of Tolokers, the process is fast and efficient.\"}),/*#__PURE__*/e(\"h2\",{children:\"Why we prioritize prompt diversity\"}),/*#__PURE__*/e(\"p\",{children:\"We\u2019ve collected over 6000 prompts for creative writing. But what\u2019s impressive is the diversity in this dataset. Thousands of Tolokers from 69 different countries with a multitude of backgrounds and cultural contexts have contributed ideas that are meaningful to them.\"}),/*#__PURE__*/e(\"p\",{children:\"A model that is trained on diverse data can emulate a broader range of human experiences and expectations, making it less biased and more helpful to real people in future applications.\"}),/*#__PURE__*/t(\"p\",{children:[\"Looking to collect prompts for your own LLM? Tap into the Toloka crowd for a global perspective. \",/*#__PURE__*/e(a,{href:\"https://toloka.ai/large-language-models/\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(n.a,{children:\"Reach out\"})}),\" to discuss the ideal pipeline for your project.\"]})]});export const richText5=/*#__PURE__*/t(i.Fragment,{children:[/*#__PURE__*/e(\"h2\",{children:\"How to build a pipeline to achieve superhuman quality  \"}),/*#__PURE__*/e(\"p\",{children:\"You may have heard that Large Language Models (LLMs) are faster, cheaper, and better than humans at text annotation. Does this mean we no longer need human data labeling? Not exactly. \"}),/*#__PURE__*/e(\"p\",{children:\"Based on our experience using LLMs on real-world text annotation projects, even the latest state-of-the-art models aren\u2019t meeting quality expectations. What\u2019s more, these models aren\u2019t always cheaper than data labeling with human annotators.\"}),/*#__PURE__*/t(\"p\",{children:[\"But we\u2019ve found that it is possible to elevate data quality by using an \",/*#__PURE__*/e(a,{href:\"https://toloka.ai/labeling-with-llms/\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(n.a,{children:\"optimal mix of human and LLM labeling\"})}),\".\"]}),/*#__PURE__*/e(\"h2\",{children:\"Comparing the quality of LLMs and human labeling\"}),/*#__PURE__*/e(\"p\",{children:\"There are a growing number of general-use LLMs publicly available, and they all belong to one of two camps: open source or closed API. Open source models are generally much cheaper to run.\"}),/*#__PURE__*/t(\"p\",{children:[\"You can see comparisons of overall model performance on Meta\u2019s \",/*#__PURE__*/e(a,{href:\"https://ai.meta.com/llama/\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(n.a,{children:\"benchmarks table\"})}),\", in \",/*#__PURE__*/e(a,{href:\"https://arxiv.org/pdf/2307.09288.pdf\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(n.a,{children:\"this paper\"})}),\", and on the \",/*#__PURE__*/e(a,{href:\"https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(n.a,{children:\"LMSYS leaderboard\"})}),\". However, most evaluation projects are based on open source datasets. To get a clear picture of LLM performance, we need to compare output on real-world projects as well.\"]}),/*#__PURE__*/e(\"p\",{children:\"We\u2019ve been testing multiple LLMs on our own data labeling projects and comparing them to human labeling with a crowd of trained annotators. To assess the quality of both humans and models, we compare their labels to ground truth labels prepared by experts.\"}),/*#__PURE__*/e(\"p\",{children:\"For most of the projects we run, we see good results from two models in particular:\"}),/*#__PURE__*/t(\"ol\",{children:[/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"GPT-4 from OpenAI (closed API)\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Llama 2 from Meta (open source) \"})})]}),/*#__PURE__*/e(\"h2\",{children:\"The dilemma of optimizing costs vs quality\"}),/*#__PURE__*/e(\"p\",{children:\"If we\u2019re talking about an open source model (Llama 2) where we only pay for GPU usage, data labeling might be hundreds of times cheaper than human crowd labeling. But what\u2019s the tradeoff? It all depends on the complexity of the task.\"}),/*#__PURE__*/e(\"p\",{children:\"Our experiments have achieved near-human accuracy with Llama 2 on sentiment analysis, spam detection, and a few other types of tasks. The inference price is generally low, but there are some other expenses involved. You need to have the infrastructure to run fine-tuning and inference, and you need to collect a human-labeled dataset for fine-tuning at the outset.\"}),/*#__PURE__*/e(\"p\",{children:\"GPT-4 provides acceptable results right out of the gate. You just need to write a detailed prompt with task instructions and examples in text format. However, the cost of inference is either more expensive or only slightly cheaper than human labeling \u2014 and the quality is lower.\"}),/*#__PURE__*/e(\"p\",{children:\"The good news is you don\u2019t have to choose between using a model or using human annotation, and you don\u2019t have to sacrifice quality to gain efficiency. We get the best results when we maximize the capabilities of human labeling and LLM labeling at the same time.\"}),/*#__PURE__*/t(\"p\",{children:[\"By carefully analyzing output, we often find subsets of data where a fine-tuned LLM performs better than humans. We can take advantage of this strength to achieve \",/*#__PURE__*/e(\"strong\",{children:\"overall quality that is better than human-only or LLM-only labeling\"}),\".\"]}),/*#__PURE__*/e(\"p\",{children:\"Does this mean we have a silver bullet to enhance quality without spending more money? Absolutely.\"}),/*#__PURE__*/e(\"h2\",{children:\"The silver bullet: intelligent hybrid pipelines\"}),/*#__PURE__*/e(\"p\",{children:\"Toloka uses hybrid pipelines, meaning we add an LLM to the data labeling pipeline alongside humans. When done correctly, this approach can optimize costs while achieving unprecedented quality.\"}),/*#__PURE__*/e(\"p\",{children:\"So how do we structure a hybrid pipeline?\"}),/*#__PURE__*/t(\"p\",{children:[\"When LLMs output a label, they can also output the \",/*#__PURE__*/e(a,{href:\"https://www.kdnuggets.com/the-quest-for-model-confidence-can-you-trust-a-black-box\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(n.a,{children:\"confidence level\"})}),\". We can use this information in a pipeline that combines LLM and human labeling. The approach is simple: the LLM labels the data, and labels with low confidence are sent to the crowd for relabeling.\"]}),/*#__PURE__*/e(\"p\",{children:\"We can change the confidence threshold to adjust the amount of data that is relabeled by humans and control the quality of the final labels.\"}),/*#__PURE__*/e(\"img\",{alt:\"Quality vs relabeling ratio. Image by author\",className:\"framer-image\",height:\"796\",src:\"https://framerusercontent.com/images/52ByVvLoy5DUTtqYByxH8505U.jpeg\",srcSet:\"https://framerusercontent.com/images/52ByVvLoy5DUTtqYByxH8505U.jpeg?scale-down-to=512 512w,https://framerusercontent.com/images/52ByVvLoy5DUTtqYByxH8505U.jpeg?scale-down-to=1024 1024w,https://framerusercontent.com/images/52ByVvLoy5DUTtqYByxH8505U.jpeg?scale-down-to=2048 2048w,https://framerusercontent.com/images/52ByVvLoy5DUTtqYByxH8505U.jpeg 2304w\",style:{aspectRatio:\"2304 / 1592\"},width:\"1152\"}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"Quality vs relabeling ratio. Image by author  \"})}),/*#__PURE__*/e(\"p\",{children:\"For typical tasks, the more we use human labeling, the higher the cost. The trick is to find the right threshold for model confidence to optimize for quality or cost, depending on our goal. We can choose any point on this curve and achieve the cost-quality trade-off that is needed for a specific task.\"}),/*#__PURE__*/e(\"p\",{children:\"In some cases, quality optimization with LLMs just doesn\u2019t work. This usually happens on complex tasks where the model\u2019s quality lags far behind human quality. We can still use the model for cost optimization and get results that are very close to human accuracy, as shown below.\"}),/*#__PURE__*/e(\"img\",{alt:\" optimization and get results that are very close to human accuracy, as shown below.\",className:\"framer-image\",height:\"796\",src:\"https://framerusercontent.com/images/GNrOOxpAx958DaxIUsMSl6O5nI.jpeg\",srcSet:\"https://framerusercontent.com/images/GNrOOxpAx958DaxIUsMSl6O5nI.jpeg?scale-down-to=512 512w,https://framerusercontent.com/images/GNrOOxpAx958DaxIUsMSl6O5nI.jpeg?scale-down-to=1024 1024w,https://framerusercontent.com/images/GNrOOxpAx958DaxIUsMSl6O5nI.jpeg?scale-down-to=2048 2048w,https://framerusercontent.com/images/GNrOOxpAx958DaxIUsMSl6O5nI.jpeg 2256w\",style:{aspectRatio:\"2256 / 1593\"},width:\"1128\"}),/*#__PURE__*/e(\"h2\",{children:\"Examples of cost-optimized pipelines\"}),/*#__PURE__*/e(\"p\",{children:\"Here are some real-life results of using cost-optimized hybrid pipelines, compared to human-only and LLM-only workflows.\"}),/*#__PURE__*/e(\"img\",{alt:\"*Cost reduction includes only human labor and inference costs. It does not reflect overall savings per project.\",className:\"framer-image\",height:\"561\",src:\"https://framerusercontent.com/images/SD0iuUya8s1xlMa72goFZ8W2W7g.png\",srcSet:\"https://framerusercontent.com/images/SD0iuUya8s1xlMa72goFZ8W2W7g.png?scale-down-to=512 512w,https://framerusercontent.com/images/SD0iuUya8s1xlMa72goFZ8W2W7g.png?scale-down-to=1024 1024w,https://framerusercontent.com/images/SD0iuUya8s1xlMa72goFZ8W2W7g.png?scale-down-to=2048 2048w,https://framerusercontent.com/images/SD0iuUya8s1xlMa72goFZ8W2W7g.png 2714w\",style:{aspectRatio:\"2714 / 1122\"},width:\"1357\"}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"*Cost reduction includes only human labor and inference costs. It does not reflect overall savings per project.\"})}),/*#__PURE__*/e(\"p\",{children:\"The potential for optimization strongly depends on the complexity of the task. Naturally, the further the model's output lags behind human quality, the less we can automate.\"}),/*#__PURE__*/e(\"h2\",{children:\"Examples of quality-optimized pipelines\"}),/*#__PURE__*/e(\"p\",{children:\"This table shows results on quality-optimized pipelines, where we leveraged the LLM to achieve better-than-human quality.\"}),/*#__PURE__*/e(\"img\",{alt:\"*Cost reduction includes only human labor and inference costs. It does not reflect overall savings per project.\",className:\"framer-image\",height:\"409\",src:\"https://framerusercontent.com/images/w4VyUvnKVXBNpKeGJjPqj8YdSs.png\",srcSet:\"https://framerusercontent.com/images/w4VyUvnKVXBNpKeGJjPqj8YdSs.png?scale-down-to=512 512w,https://framerusercontent.com/images/w4VyUvnKVXBNpKeGJjPqj8YdSs.png?scale-down-to=1024 1024w,https://framerusercontent.com/images/w4VyUvnKVXBNpKeGJjPqj8YdSs.png?scale-down-to=2048 2048w,https://framerusercontent.com/images/w4VyUvnKVXBNpKeGJjPqj8YdSs.png 2714w\",style:{aspectRatio:\"2714 / 818\"},width:\"1357\"}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"*Cost reduction includes only human labor and inference costs. It does not reflect overall savings per project.\"})}),/*#__PURE__*/e(\"p\",{children:\"On these tasks, the optimized hybrid pipeline performed impressively well compared to humans alone, while also reducing costs.\"}),/*#__PURE__*/e(\"p\",{children:\"Here's how we achieve superhuman accuracy:\"}),/*#__PURE__*/t(\"ol\",{children:[/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Fine-tune models using high-quality datasets labeled by domain experts.\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Strategically choose the confidence threshold for labeling with LLMs, and use human labeling to close any gaps.\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Identify the subset of data where LLM output is more accurate than human labeling \u2014 the bigger the subset, the more cost savings are possible while meeting high quality standards.\"})})]}),/*#__PURE__*/e(\"h2\",{children:\"Human + LLM pipelines in production\"}),/*#__PURE__*/e(\"p\",{children:\"Reliable quality is our number one priority, and we build quality control steps into every data labeling pipeline that goes into production. We check up to 5% of the final labels \u2014 both LLM output and human output \u2014 using validation by experts. We then use the results to build metrics that help us continually monitor the overall quality of the pipeline.\"}),/*#__PURE__*/e(\"p\",{children:\"So how do we mitigate quality issues? If quality drops, the standard solution is to add more data and fine-tune the model again. Sometimes the data keeps getting more complex and we don\u2019t see improvement in the model\u2019s output. In this case, we can adjust the confidence threshold for the model so that a smaller percentage of data is labeled by the model, and more data is labeled by humans.\"}),/*#__PURE__*/e(\"img\",{alt:\"Hybrid data labeling pipeline. Image by author\",className:\"framer-image\",height:\"1342\",src:\"https://framerusercontent.com/images/UUKelMvxDapOT2lzjvKC70iu2A.jpeg\",srcSet:\"https://framerusercontent.com/images/UUKelMvxDapOT2lzjvKC70iu2A.jpeg?scale-down-to=512 512w,https://framerusercontent.com/images/UUKelMvxDapOT2lzjvKC70iu2A.jpeg?scale-down-to=1024 1024w,https://framerusercontent.com/images/UUKelMvxDapOT2lzjvKC70iu2A.jpeg?scale-down-to=2048 2048w,https://framerusercontent.com/images/UUKelMvxDapOT2lzjvKC70iu2A.jpeg?scale-down-to=4096 4096w,https://framerusercontent.com/images/UUKelMvxDapOT2lzjvKC70iu2A.jpeg 4172w\",style:{aspectRatio:\"4172 / 2684\"},width:\"2086\"}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"Hybrid data labeling pipeline. Image by author\"})}),/*#__PURE__*/e(\"p\",{children:\"With the right expertise for fine-tuning and the perfect threshold to balance LLM and human labeling, hybrid pipelines can produce exceptional quality in many cases.\"}),/*#__PURE__*/e(\"h2\",{children:\"Preparing your own pipeline: paving the way to success\"}),/*#__PURE__*/e(\"p\",{children:\"To run your own hybrid data labeling pipeline that leverages LLMs and human insight, you\u2019ll need to lay the groundwork by covering 4 essential steps:\"}),/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"1.  A golden set\"}),\": To improve the quality of LLM output on your specific task, you need an accurate expert-labeled dataset for fine-tuning. The more data you have and the higher the quality, the better your fine-tuned model will perform \u2014 and the more you\u2019ll be able to automate data labeling. You can rely on qualified annotators from the \",/*#__PURE__*/e(a,{href:\"https://toloka.ai/data-labeling-platform/\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(n.a,{children:\"Toloka crowd\"})}),\" to collect high-quality training data.\"]}),/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"2.  Infrastructure for model training and inference\"}),\" and ML expertise for working with LLMs. Effective GPU utilization is crucial to make model labeling as cost-effective as possible. You will need:\"]}),/*#__PURE__*/t(\"ul\",{style:{\"--framer-font-size\":\"18px\",\"--framer-text-alignment\":\"start\",\"--framer-text-color\":\"rgb(30, 33, 38)\",\"--framer-text-stroke-width\":\"0px\",\"--framer-text-transform\":\"none\"},children:[/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Cloud instance with multiple GPUs\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Code for model fine-tuning and inference\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Storage and versioning of model weights\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Tools for model validation and quality monitoring\"})})]}),/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"3.  An annotator workforce\"}),\" available on demand to handle labels with low confidence or complex tasks. Toloka\u2019s global crowd is available 24/7 with annotators in every time zone.\"]}),/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"4. Quality control strategy\"}),\". Monitor the quality of output with ongoing human verification. Incorrect labels are sent for relabeling. At Toloka, we use advanced quality control techniques for human and LLM labeling, with continuous quality monitoring.\"]}),/*#__PURE__*/e(\"p\",{children:\"Toloka can help you in every stage of the AI development process. From finding the right LLM for your task, to taking care of fine-tuning, and designing a hybrid pipeline tailored to your needs \u2014 our team is here to support you every step of the way. The decision to optimize for quality or cost is always up to you!\"}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(a,{href:\"https://toloka.ai/talk-to-us/\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(n.a,{children:\"Talk to Toloka's AI experts\"})})})]});export const richText6=/*#__PURE__*/t(i.Fragment,{children:[/*#__PURE__*/e(\"p\",{children:\"The digital landscape grows, so does the volume of user-generated content (UGC). However, it's not always positive. Quite often instead of cheerful beautiful pictures we can stumble upon unpleasant, repulsive, and inappropriate content on social media platforms.\"}),/*#__PURE__*/e(\"p\",{children:\"To ensure safe and enjoyable online experience for all, content moderation services come in to help monitor, filter, and control the content that is shared on the internet.\"}),/*#__PURE__*/e(\"h2\",{children:\"Meaning of Content Moderation\"}),/*#__PURE__*/e(\"p\",{children:\"The implementation of content moderation ensures that offensive, obscene, or deceitful publications are removed promptly. Moderation also combats spam, trolling, phishing links, illegal content, and other forms of prohibited behavior. Modern social media platforms and websites where people can share their ideas and create user-generated content cannot exist without content moderation.\"}),/*#__PURE__*/e(\"p\",{children:\"Content moderators help to create a safe space for users to express their ideas, opinions, and creativity without fear of encountering harmful online content. Inappropriate or offensive content can take various forms, ranging from hate speech and cyberbullying to graphic imagery and misinformation. These elements not only disrupt the positive dynamics of online communities but also potentially cause emotional distress and harm to users.\"}),/*#__PURE__*/e(\"p\",{children:\"Most commonly, online platforms have community guidelines that every member of the community is obliged to abide by. Platforms are required to create and consistently apply their specific content moderation policies. Doing so will guarantee that users are informed of the rules, thus creating a fair and transparent environment. A lack of consistency in policy enforcement can lead to confusion and undermine trust in the platform's commitment to uphold a safe and respectful community.\"}),/*#__PURE__*/e(\"h2\",{children:\"Tasks and Goals of Content Moderation Services\"}),/*#__PURE__*/e(\"p\",{children:\"Each social media platform has its own moderation rules, but the basic moderating tasks and goals remain the same.\"}),/*#__PURE__*/e(\"h3\",{children:\"Identifying and Removing Inappropriate Content\"}),/*#__PURE__*/e(\"p\",{children:\"Content moderators are responsible for identifying and promptly removing content that violates community guidelines. Such undesirable data includes hate speech, harassment, graphic violence, or explicit material. Moderators oversee comments and user interactions across the platform, intervening when necessary to address inappropriate or harmful behavior, and fostering a positive online environment.\"}),/*#__PURE__*/e(\"h3\",{children:\"Implementing Age Restrictions\"}),/*#__PURE__*/e(\"p\",{children:\"Moderators may enforce age restrictions by ensuring that age-sensitive or abusive content is not accessible to users who do not meet the specified age requirements. They track and filter content to ensure that it is age-appropriate information. Content moderation involves implementing parental controls that allow parents to set limits on the content their children can access. This feature allows a parent to manage their child's digital experience.\"}),/*#__PURE__*/e(\"h3\",{children:\"Providing Security\"}),/*#__PURE__*/e(\"p\",{children:\"Content moderation protects users' personal information. By ensuring that no unauthorized sharing or misuse occurs, moderators contribute to the overall privacy and security of everyone in the digital space. From phishing attempts to online scams, content moderators are vigilant in identifying and preventing fraudulent activities that could exploit users. This helps to create a trustful and comfortable experience.\"}),/*#__PURE__*/e(\"h3\",{children:\"Maintaining Reputation and Brand Image\"}),/*#__PURE__*/e(\"p\",{children:\"Content moderation aims to identify and address incidents of hate speech and harassment. Through proactive management of these issues, moderators contribute to the social platform's reputation of inclusivity and safety. In addition to the reputation of consumer brands and the platform itself, moderators can address incidents of online harassment and cyberbullying by actively monitoring interactions and comments, preventing damage to a person's reputation in digital communities.\"}),/*#__PURE__*/e(\"h3\",{children:\"Adhering to Regional Regulations\"}),/*#__PURE__*/e(\"p\",{children:\"Content moderation implies a thorough knowledge of regional regulations governing the content of online resources. It assists compliance with legislation and monitoring violations. Social media networks are expected to comply with local and international laws related to copyright, personal data protection, human rights, and other regulations. Through awareness and adherence to these regulations, platforms ensure compliance with laws specific to the regions in which they operate. In cases where legal action is required, content moderation teams cooperate with law enforcement agencies.\"}),/*#__PURE__*/e(\"h3\",{children:\"Improving Customer Experience\"}),/*#__PURE__*/e(\"p\",{children:\"Content moderation promotes a positive online environment by making sure that interactions, comments, and posts are in line with community rules. Such a positive atmosphere contributes to a more enjoyable and friendly experience. Content moderation establishes a safe environment for users to be able to express themselves online without fear of harassment or exposure to inappropriate content.\"}),/*#__PURE__*/e(\"p\",{children:\"Moreover, content moderation encourages user engagement. While moderating content like comments and discussions, moderators create the kind of atmosphere where people are motivated to participate and socialize.\"}),/*#__PURE__*/e(\"h2\",{children:\"Types of Content Moderation\"}),/*#__PURE__*/e(\"p\",{children:\"The following are the common types of content moderation. They can be classified by the type of moderator performing the moderation and approaches to content moderation.\"}),/*#__PURE__*/e(\"h3\",{children:\"By Moderator Type\"}),/*#__PURE__*/e(\"p\",{children:\"There are 3 basic approaches to moderation:\"}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"strong\",{children:\"Manual Moderation\"})}),/*#__PURE__*/e(\"p\",{children:\"This type of content moderation requires human moderators who take part in the process of verification of UGC on digital platforms. Such moderators, often in-house or outsourced, manually evaluate content to ensure that it meets community standards and guidelines.\"}),/*#__PURE__*/e(\"p\",{children:\"As with all other types of moderation, human moderators have their pros and cons. Without a doubt, they bring a level of insight and contextual awareness that automated systems may struggle to attain, particularly in cases involving cultural nuances, sarcasm, humor, or subjective content.\"}),/*#__PURE__*/e(\"p\",{children:\"However, as the flow of user-generated content increases, the use of this type of moderation alone becomes more problematic. As the number of subscribers and volume of content grows, manually moderating each entry becomes more demanding. Hiring and training a large staff of moderators to keep up with the demands can be challenging and expensive.\"}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"strong\",{children:\"Automated Moderation\"})}),/*#__PURE__*/e(\"p\",{children:\"AI and machine learning are heavily employed to speed up and simplify the moderation process. Machine learning algorithms evaluate text, visuals, and audio within a small fraction of the time it would take humans to do it. Furthermore, AI cannot be psychologically affected by exposure to harmful content.\"}),/*#__PURE__*/e(\"p\",{children:\"AI algorithms analyze user-generated content to identify patterns of illegal or inappropriate material by following predefined rules and guidelines set by platform administrators to make decisions about content moderation. This may include turning on and setting up profanity filters in the community settings. A list of forbidden words is fed into the algorithm. The system then flags the word, replaces it with another, or blocks or rejects the entire post.\"}),/*#__PURE__*/e(\"p\",{children:\"As for the graphic content the system recognizes, deletes, or edits materials such as images, live streams, or videos by performing real-time monitoring.\"}),/*#__PURE__*/e(\"p\",{children:\"Despite the ability to effectively scale to handle large volumes of UGC, automated systems typically lack the human judgment necessary to accurately assess content with cultural or context-specific components. Among other things, AI can generate false positives by mistakenly flagging content as offensive when it is not.\"}),/*#__PURE__*/e(\"p\",{children:\"Therefore, automated content moderation cannot fully replace human oversight, especially in complex situations. This is why a combination of AI technology and the effort of human content moderators is commonly used.\"}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"strong\",{children:\"Hybrid Moderation\"})}),/*#__PURE__*/e(\"p\",{children:\"Hybrid moderation describes an approach to content supervision that combines both automated moderation through the use of artificial intelligence and moderation carried out by humans. Simply put, it can be described as follows: AI screens the content, then controversial or challenging content is submitted to a human for evaluation.\"}),/*#__PURE__*/e(\"p\",{children:\"Automated algorithms provide the scalability essential for handling massive amounts of content and apply rules consistently, while humans bring the necessary contextual understanding and the flexibility and adaptability to handle nuanced content.\"}),/*#__PURE__*/e(\"h3\",{children:\"By Moderation Approach\"}),/*#__PURE__*/e(\"p\",{children:\"These types of moderation can employ either manual or automated content moderation tools, or the two combined:\"}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"strong\",{children:\"Pre-moderation\"})}),/*#__PURE__*/e(\"p\",{children:\"In pre-moderation, user-generated content is vetted and approved or declined by content moderators before it becomes available to the public. It could be accomplished manually or with the help of automation. While this is one of the most reliable ways of blocking harmful content, it is a rather slow process and not applicable in the fast-paced online landscape.\"}),/*#__PURE__*/e(\"p\",{children:\"Although it does an excellent job of maintaining a high standard of content quality by preventing inappropriate comments or harmful material from being displayed, content approval can introduce delays in its visibility, impacting real-time interactions on the platform.\"}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"strong\",{children:\"Post-moderation\"})}),/*#__PURE__*/e(\"p\",{children:\"Post-moderation lets a user publish UGC on social media right away, however, it's moderated afterwards. Moderators examine posts or flagged content and take appropriate actions. Such an approach gives users an opportunity to post content in real-time, which encourages immediate engagement and interaction.\"}),/*#__PURE__*/e(\"p\",{children:\"With a post-moderation approach users themselves often report content that breaches the social media policy. This kind of reporting mechanism helps platforms identify and address inappropriate or harmful content faster and more effectively. The main concern about post-moderation is the fact that inappropriate or toxic content may be exposed to the audience before it is deleted. This may result in negative user experiences, damage to the platform's reputation, and even legal consequences if the posted material breaches laws or regulations.\"}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"strong\",{children:\"Reactive Moderation\"})}),/*#__PURE__*/e(\"p\",{children:\"Reactive Moderation entails self-regulation of an online platform by its participants. It may be applied on its own or implemented in conjunction with other, more reliable ways of getting rid of unwanted material.\"}),/*#__PURE__*/e(\"p\",{children:\"Content moderation efforts are initiated in response to incidents or user reports, in addition to searching for unwanted content and its moderation by humans and/or automated systems. Content moderation efforts are initiated in response to incidents or user reports, instead of the active searching for unwanted content and its moderation. This approach requires moderators or algorithms to respond to specific issues, violations, or user reports. Reactive moderation may be utilized on its own or in conjunction with pre- and post-moderation.\"}),/*#__PURE__*/e(\"p\",{children:\"The community's subscribers become responsible for the content that is shared in the community. The key benefit of this approach lies in the fact that it can be scaled as your community grows without putting additional strain on moderators. The more subscribers you have, the more users are keeping an eye on the community.\"}),/*#__PURE__*/e(\"p\",{children:\"However, sometimes it can take a long time for unwanted content to be flagged by someone or not prevent all instances of inappropriate content at all. In addition, participants may abuse reporting mechanisms, leading to false positives or misuse.\"}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"strong\",{children:\"Distributed Moderation\"})}),/*#__PURE__*/e(\"p\",{children:\"Distributed moderation refers to a relatively rare kind of moderation of UGC. It involves leveraging the collective efforts of a platform's user community in content moderation tasks. Unlike reactive moderation, this type of moderation means that the community will have complete control over the removal of undesirable materials on the site. That is, the moderation responsibility for the community rests solely on its members. Members of the platform actively participate in reporting, flagging, and sometimes even moderating content. It typically relies on a ranking mechanism that community members employ to vote on whether the content meets community policies.\"}),/*#__PURE__*/e(\"p\",{children:\"It works like this, for example, if enough people vote against a certain post, the system removes or hides this material from the website. There are no designated moderators or AIs constantly seeking out and modifying inappropriate information on such sites. Such a system works well on websites where participants are highly involved in the project. However, the website owner cannot fully control what is moderated and what is not.\"}),/*#__PURE__*/e(\"p\",{children:\"Even though it empowers users to actively shape the platform's content environment and gives them a sense of shared responsibility, some difficulties prevent this approach from becoming more widespread. The main challenge of distributed moderation is the fact that content moderation decisions may differ according to individual viewpoints, leading to consistency issues. On top of that, ensuring that the moderation decisions are valid and fair can be a difficult if not impossible task.\"}),/*#__PURE__*/e(\"p\",{children:\"More frequently, platforms employ a mix of these moderation approaches to create a cohesive and successful content moderation strategy. There are advantages and challenges to each type of content moderation, and the choice often depends on the type of platform, the amount of user-generated content, and the level of oversight and engagement desired.\"}),/*#__PURE__*/e(\"h2\",{children:\"Content Moderation Tools\"}),/*#__PURE__*/e(\"p\",{children:\"Diverse content moderation tools and platforms exist to cater to different needs and preferences. Some popular content moderation tools are listed below:\"}),/*#__PURE__*/t(\"ul\",{style:{\"--framer-font-size\":\"18px\",\"--framer-text-alignment\":\"start\",\"--framer-text-color\":\"rgb(30, 33, 38)\",\"--framer-text-stroke-width\":\"0px\",\"--framer-text-transform\":\"none\"},children:[/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Amazon Rekognition. It is an AI service that provides image and video analysis. It can be employed for content moderation, identifying explicit or suggestive content in images and videos;\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Hive Moderation. It offers automated content moderation solutions with a focus on achieving human-level accuracy. The tool supports the moderation of visual content, including images and GIFs, text and audio moderation;\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Respondology. This platform is specially designed to moderate content that contains racist remarks, insults, and other types of profanity. Such a targeted approach is valuable for brands and organizations striving to maintain a positive and respectful online environment;\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Text Moderation powered by PaLM 2. Text Moderation scans text content to identify and filter out potentially offensive or harmful material, actively working to create a secure online space. It filters and moderates text outputs generated by AI models, ensuring they adhere to ethical standards and do not produce content that could be offensive or harmful;\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Azure AI Content Moderation. It includes features for text moderation, image moderation, and video moderation. It can detect and filter content related to adult content, offensive language, and more.\"})})]}),/*#__PURE__*/e(\"h2\",{children:\"Solutions that Simplify the Task of Content Moderation\"}),/*#__PURE__*/e(\"p\",{children:\"Manual content moderation is an outdated and tedious process. When there is a lot of information, and the amount of user-generated content has multiplied manyfold in recent decades, manual human moderation alone is not just insufficient, it simply becomes ineffective.\"}),/*#__PURE__*/e(\"p\",{children:\"Recently, this crucial task of maintaining a safe and positive online environment on online platforms has undergone a transformational change with the integration of artificial intelligence and machine learning into human moderation.\"}),/*#__PURE__*/e(\"h3\",{children:\"Toloka's Moderation Solution: AI Moderation with Human-in-the-loop Quality Control\"}),/*#__PURE__*/e(\"p\",{children:\"Toloka's content moderation platform merges the efficiency of AI and machine learning with valuable human input. Offering an implementation in just one day, Toloka ensures a quick and efficient transition to automated content moderation, minimizing disruption to day-to-day operations and providing smooth integration of artificial intelligence technologies into the business process framework.\"}),/*#__PURE__*/e(\"p\",{children:\"Toloka takes into account the diversity of user-generated content by offering moderation solutions for all types of data. This multifaceted approach enables companies to maintain a secure and inclusive online environment for different types of content. Content moderation solutions from Toloka include:\"}),/*#__PURE__*/t(\"ul\",{style:{\"--framer-font-size\":\"18px\",\"--framer-text-alignment\":\"start\",\"--framer-text-color\":\"rgb(30, 33, 38)\",\"--framer-text-stroke-width\":\"0px\",\"--framer-text-transform\":\"none\"},children:[/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Image moderation. Toloka's image moderation services are designed to identify and flag images that contain illegal content, such as images promoting violence, terrorism, or any other criminal activities, and images with adult themes. The solution also addresses copyright issues by identifying copyright-infringing images;\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Text moderation. The system monitors and controls written content, inspects and filters texts for such elements as hate speech, offensive and insulting language, and misleading content, intending to create a safer and more respectful cyberspace;\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Video moderation. In addition to addressing explicit or offensive material in various videos or live streams, video moderation also addresses copyright concerns by identifying and managing content that infringes on intellectual property rights. Moreover, Toloka's system seeks out and restricts content that violates social media policies or legal regulations;\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Audio moderation. The audio content in 13 different languages can be converted into written text using natural language processing techniques. The transcribed and processed text is then subjected to moderation which helps to identify and flag content that violates community rules or legal provisions, for instance, hate speech, explicit content, or any other prohibited material, etc.\"})})]}),/*#__PURE__*/e(\"p\",{children:\"Toloka's moderation system extends beyond basic analysis to detect explicit, implicit, or perceived threats of violence in advance. It improves security by allowing immediate action to be taken against undesirable content that hints at physical harm, aggression, or intimidation in texts, audio, or images and videos.\"}),/*#__PURE__*/e(\"p\",{children:\"The solution excels at identifying and handling derogatory language. Toloka's solution is finely tuned to detect disrespectful, offensive, or hateful language directed at individuals or groups, ensuring that people are treated with respect. The platform also targets vulgar language and offensive expressions, maintaining a space free from inappropriate or indecent content and any kind of online harassment and cyberbullying.\"}),/*#__PURE__*/e(\"p\",{children:\"Toloka's moderation solution involves protecting user privacy by identifying and dealing with sensitive or identifiable information about individuals, such as full names, addresses, telephone numbers, and other personal data.\"}),/*#__PURE__*/e(\"p\",{children:\"In addition to the above-mentioned classes, it allows the recognition and concealment of nonsensical texts, explicit materials, spam advertising and promotions in any kind of content, be it visual, textual, or auditory.\"}),/*#__PURE__*/e(\"h3\",{children:\"Overseeing the Process of Moderation\"}),/*#__PURE__*/e(\"p\",{children:\"Overseeing content moderation with Toloka's solution is easy, as there are several thoughtful solutions for this.\"}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"strong\",{children:\"Convenient Statistics Panel\"})}),/*#__PURE__*/e(\"p\",{children:\"Toloka customers have access to an online dashboard that displays statistics related to content moderation. This panel presents an overview of moderation tasks by day, showing metrics such as the number of decisions made for processed content, the percentage of manual and automated moderation, the number of content moderation decisions approved and rejected, and other important data. Toloka's analytics feature allows a deeper insight into moderation results by providing visualized statistics organized by category.\"}),/*#__PURE__*/e(\"p\",{children:\"The platform enables a user to fine-tune their moderation settings to increase accuracy. This can include manual moderation for ambiguous content and edge cases. By giving consumers the ability to set their own parameters, Toloka allows them to configure the moderation process based on the specific needs and nuances of the content in question. Besides, Toloka offers a user-friendly interface that can be accessed both online and on mobile devices.\"}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"strong\",{children:\"Prompt Real-Time Monitoring\"})}),/*#__PURE__*/e(\"p\",{children:\"The Toloka platform offers real-time overseeing, allowing customers to observe and assess all comments in their social stream as they happen. This feature ensures that they stay up-to-date with the latest interactions and content moderation in real-time.\"}),/*#__PURE__*/e(\"p\",{children:\"Access to a granular view of moderation results lets you get the details of what's being blocked and why. Toloka lets you dive into the details of moderation results so you know what's behind each comment that was blocked.\"}),/*#__PURE__*/e(\"p\",{children:\"The moderation tool allows to categorize and sort those blocked comments by type. This categorization system helps in organizing and understanding the nature of the content being flagged or blocked. Users can group comments under different categories such as Racism, LGBTQ+ Slurs, Severe Swearing, etc.\"}),/*#__PURE__*/e(\"h2\",{children:\"Significance of Content Moderation Services\"}),/*#__PURE__*/e(\"p\",{children:\"Wrapping things up, it's clear that content moderation is the backbone of our online experience. It makes sure that everyone has a good time online without getting traumatized in the process. From a business perspective, integrating content moderation services is about ensuring the longevity and sustainability of a brand in the digital scene.\"}),/*#__PURE__*/e(\"p\",{children:\"Businesses and online communities gain significant benefits from implementing effective content moderation practices. Organizations build a positive brand image, user trust, and attract a wider audience by ensuring their platforms are free of offensive or unlawful content. Content moderation also contributes to legal risk mitigation and adherence to regulatory requirements by keeping platforms operating within acceptable guidelines.\"}),/*#__PURE__*/e(\"p\",{children:\"Especially in a world where digital communication shapes perceptions and influences decision-making, companies that focus on content moderation are not just addressing reputational risks. They build the basis for lasting success in the constantly changing online landscape.\"})]});\nexport const __FramerMetadata__ = {\"exports\":{\"richText4\":{\"type\":\"variable\",\"annotations\":{\"framerContractVersion\":\"1\"}},\"richText5\":{\"type\":\"variable\",\"annotations\":{\"framerContractVersion\":\"1\"}},\"richText3\":{\"type\":\"variable\",\"annotations\":{\"framerContractVersion\":\"1\"}},\"richText6\":{\"type\":\"variable\",\"annotations\":{\"framerContractVersion\":\"1\"}},\"richText\":{\"type\":\"variable\",\"annotations\":{\"framerContractVersion\":\"1\"}},\"richText2\":{\"type\":\"variable\",\"annotations\":{\"framerContractVersion\":\"1\"}},\"richText1\":{\"type\":\"variable\",\"annotations\":{\"framerContractVersion\":\"1\"}},\"__FramerMetadata__\":{\"type\":\"variable\"}}}"],
  "mappings": "6JAAsJ,IAAMA,EAAsBC,EAAIC,EAAS,CAAC,SAAS,CAAcC,EAAE,IAAI,CAAC,SAAS,2PAA2P,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,iBAA8BE,EAAE,SAAS,CAAC,SAAS,mBAAmB,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,6BAA6B,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,oQAAoQ,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,kSAAkS,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,8WAA8W,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,qIAAqI,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,4GAA4G,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,4CAA4C,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,4EAA4E,CAAC,EAAeF,EAAE,KAAK,CAAC,SAAS,CAAcE,EAAE,KAAK,CAAC,kBAAkB,IAAI,MAAM,CAAC,qBAAqB,OAAO,sBAAsB,eAAe,2BAA2B,MAAM,EAAE,SAAsBA,EAAE,IAAI,CAAC,SAAS,4HAA4H,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,MAAM,CAAC,qBAAqB,OAAO,sBAAsB,eAAe,2BAA2B,MAAM,EAAE,SAAsBA,EAAE,IAAI,CAAC,SAAS,wEAAwE,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,gbAAgb,CAAC,EAAeF,EAAE,KAAK,CAAC,SAAS,CAAcE,EAAE,KAAK,CAAC,kBAAkB,IAAI,MAAM,CAAC,qBAAqB,OAAO,sBAAsB,eAAe,2BAA2B,MAAM,EAAE,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,UAAU,CAAC,EAAE,kKAA+KA,EAAE,SAAS,CAAC,SAAS,UAAU,CAAC,EAAE,iLAAiL,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,MAAM,CAAC,qBAAqB,OAAO,sBAAsB,eAAe,2BAA2B,MAAM,EAAE,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,uBAAuB,CAAC,EAAE,wLAAwL,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,MAAM,CAAC,qBAAqB,OAAO,sBAAsB,eAAe,2BAA2B,MAAM,EAAE,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAC,gBAA6BE,EAAE,SAAS,CAAC,SAAS,YAAY,CAAC,EAAE,oPAAoP,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,MAAM,CAAC,qBAAqB,OAAO,sBAAsB,eAAe,2BAA2B,MAAM,EAAE,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAC,2FAAwGE,EAAE,SAAS,CAAC,SAAS,YAAY,CAAC,EAAE,kKAAkK,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,MAAM,CAAC,qBAAqB,OAAO,sBAAsB,eAAe,2BAA2B,MAAM,EAAE,SAAsBA,EAAE,IAAI,CAAC,SAAS,4PAA4P,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,MAAM,CAAC,qBAAqB,OAAO,sBAAsB,eAAe,2BAA2B,MAAM,EAAE,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAC,yCAAsDE,EAAE,SAAS,CAAC,SAAS,qCAAqC,CAAC,EAAE,sEAAsE,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,wGAAwG,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,0LAA0L,CAAC,EAAeF,EAAE,KAAK,CAAC,SAAS,CAAcE,EAAE,KAAK,CAAC,kBAAkB,IAAI,MAAM,CAAC,qBAAqB,OAAO,sBAAsB,eAAe,2BAA2B,MAAM,EAAE,SAAsBA,EAAE,IAAI,CAAC,SAAS,wDAAwD,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,MAAM,CAAC,qBAAqB,OAAO,sBAAsB,eAAe,2BAA2B,MAAM,EAAE,SAAsBA,EAAE,IAAI,CAAC,SAAS,2DAA2D,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,MAAM,CAAC,qBAAqB,OAAO,sBAAsB,eAAe,2BAA2B,MAAM,EAAE,SAAsBA,EAAE,IAAI,CAAC,SAAS,kCAAkC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,gDAAgD,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,8EAA8E,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,otBAAotB,CAAC,EAAeF,EAAE,KAAK,CAAC,SAAS,CAAcE,EAAE,KAAK,CAAC,kBAAkB,IAAI,MAAM,CAAC,qBAAqB,OAAO,sBAAsB,eAAe,2BAA2B,MAAM,EAAE,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAEC,EAAE,CAAC,KAAK,uBAAuB,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,UAAU,CAAC,CAAC,CAAC,EAAE,6TAA0UF,EAAEC,EAAE,CAAC,KAAK,0CAA0C,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,OAAO,CAAC,CAAC,CAAC,EAAE,oEAAoE,CAAC,CAAC,CAAC,CAAC,EAAeF,EAAE,KAAK,CAAC,kBAAkB,IAAI,MAAM,CAAC,qBAAqB,OAAO,sBAAsB,eAAe,2BAA2B,MAAM,EAAE,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAEC,EAAE,CAAC,KAAK,0DAA0D,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,WAAW,CAAC,CAAC,CAAC,EAAE,oNAAoN,CAAC,CAAC,CAAC,CAAC,EAAeF,EAAE,KAAK,CAAC,kBAAkB,IAAI,MAAM,CAAC,qBAAqB,OAAO,sBAAsB,eAAe,2BAA2B,MAAM,EAAE,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAC,uDAAoEE,EAAEC,EAAE,CAAC,KAAK,0CAA0C,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,MAAM,CAAC,CAAC,CAAC,EAAE,qEAAkFF,EAAEC,EAAE,CAAC,KAAK,4EAA4E,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,uCAAuC,CAAC,CAAC,CAAC,EAAE,6MAA6M,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeJ,EAAE,KAAK,CAAC,SAAS,CAAC,wBAAqCE,EAAEC,EAAE,CAAC,KAAK,wGAAwG,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,YAAY,CAAC,CAAC,CAAC,EAAE,gIAAgI,CAAC,CAAC,EAAeF,EAAE,KAAK,CAAC,SAAS,0CAA0C,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAEC,EAAE,CAAC,KAAK,4EAA4E,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,0BAA0B,CAAC,CAAC,CAAC,EAAE,IAAiBF,EAAEC,EAAE,CAAC,KAAK,kCAAkC,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,sBAAsB,CAAC,CAAC,CAAC,EAAE,QAAqBF,EAAEC,EAAE,CAAC,KAAK,oDAAoD,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,sCAAsC,CAAC,CAAC,CAAC,EAAE,0TAA0T,CAAC,CAAC,EAAeJ,EAAE,KAAK,CAAC,SAAS,CAAcE,EAAE,KAAK,CAAC,kBAAkB,IAAI,MAAM,CAAC,qBAAqB,OAAO,sBAAsB,eAAe,2BAA2B,MAAM,EAAE,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAEC,EAAE,CAAC,KAAK,4EAA4E,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,0BAA0B,CAAC,CAAC,CAAC,EAAE,ofAAof,CAAC,CAAC,CAAC,CAAC,EAAeF,EAAE,KAAK,CAAC,kBAAkB,IAAI,MAAM,CAAC,qBAAqB,OAAO,sBAAsB,eAAe,2BAA2B,MAAM,EAAE,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAEC,EAAE,CAAC,KAAK,kCAAkC,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,qBAAqB,CAAC,CAAC,CAAC,EAAE,kdAAkd,CAAC,CAAC,CAAC,CAAC,EAAeF,EAAE,KAAK,CAAC,kBAAkB,IAAI,MAAM,CAAC,qBAAqB,OAAO,sBAAsB,eAAe,2BAA2B,MAAM,EAAE,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAEC,EAAE,CAAC,KAAK,oDAAoD,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,yCAAyC,CAAC,CAAC,CAAC,EAAE,qJAAkKF,EAAEC,EAAE,CAAC,KAAK,uDAAuD,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,gCAAgC,CAAC,CAAC,CAAC,EAAE,oKAAoK,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeJ,EAAE,IAAI,CAAC,SAAS,CAAC,6GAA0HE,EAAEC,EAAE,CAAC,KAAK,2CAA2C,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,yBAAyB,CAAC,CAAC,CAAC,EAAE,+DAA4EF,EAAEC,EAAE,CAAC,KAAK,iDAAiD,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,OAAO,CAAC,CAAC,CAAC,EAAE,uKAAoLF,EAAEC,EAAE,CAAC,KAAK,oDAAoD,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,OAAO,CAAC,CAAC,CAAC,EAAE,4KAAyLF,EAAEC,EAAE,CAAC,KAAK,iEAAiE,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,aAAa,CAAC,CAAC,CAAC,EAAE,IAAiBF,EAAEC,EAAE,CAAC,KAAK,8CAA8C,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,WAAW,CAAC,CAAC,CAAC,EAAE,QAAqBF,EAAEC,EAAE,CAAC,KAAK,mDAAmD,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,uCAAuC,CAAC,CAAC,CAAC,EAAE,4DAA4D,CAAC,CAAC,EAAeF,EAAE,KAAK,CAAC,SAAS,eAAe,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,mUAAmU,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,mJAAmJ,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,wRAAwR,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,YAAY,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,6JAA0KE,EAAEC,EAAE,CAAC,KAAK,+BAA+B,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,YAAY,CAAC,CAAC,CAAC,EAAE,kBAAkB,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeC,EAAuBL,EAAIC,EAAS,CAAC,SAAS,CAAcC,EAAE,IAAI,CAAC,SAAS,ogBAA+f,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,+sBAA0sB,CAAC,EAAeF,EAAE,aAAa,CAAC,SAAS,CAAcE,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,kOAAkO,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,2CAAsC,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,4uBAAouB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,iTAAuS,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,wCAAwC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,6dAA6d,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,oaAAoa,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,4LAAkL,CAAC,EAAeF,EAAE,aAAa,CAAC,SAAS,CAAcE,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,yHAAoH,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,qCAAgC,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,6PAA6P,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,yDAAyD,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,gcAA2b,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,mYAAoY,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,saAAsa,CAAC,EAAeF,EAAE,aAAa,CAAC,SAAS,CAAcE,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,oHAA+G,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,2CAAsC,CAAC,CAAC,CAAC,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,wDAAqEE,EAAEC,EAAE,CAAC,KAAK,yGAAyG,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,4BAA4B,CAAC,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeE,EAAuBN,EAAIC,EAAS,CAAC,SAAS,CAAcC,EAAE,IAAI,CAAC,SAAS,oZAA+Y,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,uCAAoDE,EAAEC,EAAE,CAAC,KAAK,2CAA2C,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,uBAAuB,CAAC,CAAC,CAAC,EAAE,iTAAiT,CAAC,CAAC,EAAeF,EAAE,KAAK,CAAC,SAAS,4BAA4B,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,obAAicE,EAAEC,EAAE,CAAC,KAAK,mCAAmC,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,qDAAqD,CAAC,CAAC,CAAC,EAAE,iMAA4L,CAAC,CAAC,EAAeF,EAAE,KAAK,CAAC,SAAS,kDAAkD,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,kFAA+FE,EAAE,SAAS,CAAC,SAAS,gCAAgC,CAAC,EAAE,8QAA8Q,CAAC,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,iCAA8CE,EAAE,SAAS,CAAC,SAAS,aAAa,CAAC,EAAE,4PAA4P,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,2XAA2X,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,mbAAmb,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,2BAAwCE,EAAE,SAAS,CAAC,SAAS,YAAY,CAAC,EAAE,idAAid,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,4CAA4C,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,wDAAqEE,EAAE,SAAS,CAAC,SAAS,eAAe,CAAC,EAAE,6TAA6T,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,+RAA+R,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,8TAA8T,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,4YAA4Y,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,6PAA6P,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,ukBAAukB,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,qDAAqD,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,2YAA2Y,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,mBAAmB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,kWAAkW,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,0NAA0N,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,+VAA+V,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,0BAA0B,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,wXAAwX,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,+XAA4YE,EAAEC,EAAE,CAAC,KAAK,2CAA2C,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,uBAAuB,CAAC,CAAC,CAAC,EAAE,UAAU,CAAC,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,mgBAAmgB,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,+BAA+B,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,qRAAqR,CAAC,EAAeF,EAAE,KAAK,CAAC,MAAM,CAAC,qBAAqB,OAAO,0BAA0B,QAAQ,sBAAsB,kBAAkB,6BAA6B,MAAM,0BAA0B,MAAM,EAAE,SAAS,CAAcE,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,wCAAwC,CAAC,EAAE,gPAAgP,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,aAAa,CAAC,EAAE,6RAA6R,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,6BAA6B,CAAC,EAAE,qQAAqQ,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,uBAAuB,CAAC,EAAE,sZAAsZ,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,yBAAyB,CAAC,EAAE,+SAA+S,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,mCAAmC,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,6BAA6B,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,+QAA+Q,CAAC,EAAeF,EAAE,KAAK,CAAC,MAAM,CAAC,qBAAqB,OAAO,0BAA0B,QAAQ,sBAAsB,kBAAkB,6BAA6B,MAAM,0BAA0B,MAAM,EAAE,SAAS,CAAcE,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,iHAAiH,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,sFAAsF,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,iFAAiF,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,iHAAiH,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,kMAAkM,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,iBAAiB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,sTAAsT,CAAC,EAAeF,EAAE,KAAK,CAAC,MAAM,CAAC,qBAAqB,OAAO,0BAA0B,QAAQ,sBAAsB,kBAAkB,6BAA6B,MAAM,0BAA0B,MAAM,EAAE,SAAS,CAAcE,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,sJAAsJ,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,uEAAuE,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,uIAAuI,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,kFAAkF,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,6GAA6G,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,uJAAuJ,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,0GAA0G,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,6HAA6H,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,gZAAgZ,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,+BAA+B,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,8BAA8B,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAEC,EAAE,CAAC,KAAK,2CAA2C,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,sBAAsB,CAAC,CAAC,CAAC,EAAE,oUAAoU,CAAC,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAsBA,EAAE,SAAS,CAAC,SAAS,0CAA0C,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,igBAAigB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,SAAS,CAAC,SAAS,gEAAgE,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,yiCAAyiC,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,mCAAmC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,SAAS,CAAC,SAAS,+CAA+C,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,8bAA8b,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,SAAS,CAAC,SAAS,WAAW,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,0SAA0S,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,SAAS,CAAC,SAAS,kBAAkB,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,uaAAua,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,OAAO,CAAC,EAAE,mYAAoY,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,yCAAyC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,wSAAwS,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,SAAS,CAAC,SAAS,iEAAiE,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,oTAAoT,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,4PAA4P,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,iCAAiC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,kJAAkJ,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,cAAc,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,8pBAA8pB,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,kCAAkC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,uZAAuZ,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,wWAAwW,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,cAAc,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,ybAAyb,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,YAAY,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,qWAAqW,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,y1BAAy1B,CAAC,CAAC,CAAC,CAAC,EAAeK,EAAuBP,EAAIC,EAAS,CAAC,SAAS,CAAcC,EAAE,IAAI,CAAC,SAAS,oTAAoT,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,+TAA+T,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,yUAAyU,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,6BAA6B,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,2OAA2O,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,+bAA+b,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,oQAAoQ,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,wcAAwc,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,oUAAoU,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,kCAAkC,CAAC,EAAeF,EAAE,KAAK,CAAC,MAAM,CAAC,qBAAqB,OAAO,0BAA0B,QAAQ,sBAAsB,kBAAkB,6BAA6B,MAAM,0BAA0B,MAAM,EAAE,SAAS,CAAcE,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,8CAA8C,CAAC,EAAE,yOAAyO,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,gEAAgE,CAAC,EAAE,wPAAwP,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,6CAA6C,CAAC,EAAE,oXAAoX,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,yBAAyB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,oPAAoP,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,iBAAiB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,4iBAA4iB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,wPAAwP,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,kBAAkB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,ieAAie,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,mCAAmC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,mhBAAmhB,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,iBAAiB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,mcAAmc,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,sCAAsC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,qUAAqU,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,mSAAmS,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,0NAA0N,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,qaAAqa,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,6OAA6O,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,wEAAwE,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,+LAA+L,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,miBAAmiB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,8cAA8c,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,gUAAgU,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,oXAAoX,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,wMAAwM,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,sOAAsO,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,0CAA0C,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,mVAAmV,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,0BAA0B,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,iTAAiT,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,uPAAuP,CAAC,EAAeF,EAAE,KAAK,CAAC,MAAM,CAAC,qBAAqB,OAAO,0BAA0B,QAAQ,sBAAsB,kBAAkB,6BAA6B,MAAM,0BAA0B,MAAM,EAAE,SAAS,CAAcE,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,sBAAsB,CAAC,EAAE,2MAA2M,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,2BAA2B,CAAC,EAAE,6MAA6M,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,iBAAiB,CAAC,EAAE,+IAA+I,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,mBAAmB,CAAC,EAAE,8FAA8F,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,sBAAsB,CAAC,EAAE,oJAAoJ,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,uBAAuB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,8XAA8X,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,6NAA6N,CAAC,EAAeF,EAAE,KAAK,CAAC,MAAM,CAAC,qBAAqB,OAAO,0BAA0B,QAAQ,sBAAsB,kBAAkB,6BAA6B,MAAM,0BAA0B,MAAM,EAAE,SAAS,CAAcE,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,mLAAmL,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,+NAA+N,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,2CAA2C,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,yVAAyV,CAAC,EAAeF,EAAE,KAAK,CAAC,MAAM,CAAC,qBAAqB,OAAO,0BAA0B,QAAQ,sBAAsB,kBAAkB,6BAA6B,MAAM,0BAA0B,MAAM,EAAE,SAAS,CAAcE,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,wOAAwO,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,0LAA0L,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,sKAAsK,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,gCAAgC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,siBAAsiB,CAAC,EAAeF,EAAE,KAAK,CAAC,MAAM,CAAC,qBAAqB,OAAO,0BAA0B,QAAQ,sBAAsB,kBAAkB,6BAA6B,MAAM,0BAA0B,MAAM,EAAE,SAAS,CAAcE,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,2IAA2I,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,2JAA2J,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,iIAAiI,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,sCAAsC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,2eAA2e,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,qYAAqY,CAAC,EAAeF,EAAE,KAAK,CAAC,MAAM,CAAC,qBAAqB,OAAO,0BAA0B,QAAQ,sBAAsB,kBAAkB,6BAA6B,MAAM,0BAA0B,MAAM,EAAE,SAAS,CAAcE,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,mPAAmP,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,mKAAmK,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,uOAAuO,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,wCAAwC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,8eAA8e,CAAC,EAAeF,EAAE,KAAK,CAAC,MAAM,CAAC,qBAAqB,OAAO,0BAA0B,QAAQ,sBAAsB,kBAAkB,6BAA6B,MAAM,0BAA0B,MAAM,EAAE,SAAS,CAAcE,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,yPAAyP,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,2NAA2N,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,oMAAoM,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,oBAAoB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,4UAA4U,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,mYAAmY,CAAC,EAAeF,EAAE,KAAK,CAAC,MAAM,CAAC,qBAAqB,OAAO,0BAA0B,QAAQ,sBAAsB,kBAAkB,6BAA6B,MAAM,0BAA0B,MAAM,EAAE,SAAS,CAAcE,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,gKAAgK,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,0IAA0I,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,gLAAgL,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,+CAA+C,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,6TAA6T,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,aAAa,CAAC,EAAE,yTAAyT,CAAC,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,iBAAiB,CAAC,EAAE,+LAA+L,CAAC,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,cAAc,CAAC,EAAE,mNAAmN,CAAC,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,qBAAqB,CAAC,EAAE,kJAAkJ,CAAC,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,uBAAuB,CAAC,EAAE,qMAAqM,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,kCAAkC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,kLAAkL,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,YAAY,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,+WAA+W,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,kZAAkZ,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,YAAY,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,maAAma,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,gTAAgT,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,eAAe,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,6lBAA6lB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,oQAAoQ,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,WAAW,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,wfAAwf,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,sPAAsP,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,6VAA6V,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,YAAY,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,2QAA2Q,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,sOAAsO,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,+VAA+V,CAAC,CAAC,CAAC,CAAC,EAAeM,EAAuBR,EAAIC,EAAS,CAAC,SAAS,CAAcC,EAAE,IAAI,CAAC,SAAS,kVAAkV,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,0EAA0E,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,4MAAuM,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,QAAqBE,EAAEC,EAAE,CAAC,KAAK,qDAAqD,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,aAAa,CAAC,CAAC,CAAC,EAAE,4UAAkU,CAAC,CAAC,EAAeF,EAAE,KAAK,CAAC,SAAS,2CAAsC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,kRAAwQ,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,qQAAqQ,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,gDAAgD,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,wHAAqIE,EAAEC,EAAE,CAAC,KAAK,sDAAsD,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,UAAU,CAAC,CAAC,CAAC,EAAE,kRAA6Q,CAAC,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,ycAA+b,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,0BAA0B,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,wmBAA8lB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,siBAAmgB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,kEAAkE,CAAC,EAAeA,EAAE,MAAM,CAAC,IAAI,8BAA8B,UAAU,eAAe,OAAO,MAAM,IAAI,uEAAuE,OAAO,uQAAuQ,MAAM,CAAC,YAAY,aAAa,EAAE,MAAM,KAAK,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,6BAA6B,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,8CAA8C,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,2bAAua,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,2EAA2E,CAAC,EAAeA,EAAE,MAAM,CAAC,IAAI,2BAA2B,UAAU,eAAe,OAAO,MAAM,IAAI,wEAAwE,OAAO,0QAA0Q,MAAM,CAAC,YAAY,aAAa,EAAE,MAAM,KAAK,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,0BAA0B,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,sDAAsD,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,6TAAyS,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,2SAAiS,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,iXAAwV,CAAC,EAAeA,EAAE,MAAM,CAAC,IAAI,iCAAiC,UAAU,eAAe,OAAO,MAAM,IAAI,wEAAwE,OAAO,4KAA4K,MAAM,CAAC,YAAY,aAAa,EAAE,MAAM,KAAK,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,gCAAgC,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,0eAAge,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,oCAAoC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,uRAA6Q,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,0LAA0L,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,oGAAiHE,EAAEC,EAAE,CAAC,KAAK,2CAA2C,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,WAAW,CAAC,CAAC,CAAC,EAAE,kDAAkD,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeK,EAAuBT,EAAIC,EAAS,CAAC,SAAS,CAAcC,EAAE,KAAK,CAAC,SAAS,yDAAyD,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,0LAA0L,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,kQAAmP,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,gFAAwFE,EAAEC,EAAE,CAAC,KAAK,wCAAwC,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,uCAAuC,CAAC,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC,EAAeF,EAAE,KAAK,CAAC,SAAS,kDAAkD,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,8LAA8L,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,uEAA+EE,EAAEC,EAAE,CAAC,KAAK,6BAA6B,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,kBAAkB,CAAC,CAAC,CAAC,EAAE,QAAqBF,EAAEC,EAAE,CAAC,KAAK,uCAAuC,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,YAAY,CAAC,CAAC,CAAC,EAAE,gBAA6BF,EAAEC,EAAE,CAAC,KAAK,gEAAgE,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,mBAAmB,CAAC,CAAC,CAAC,EAAE,6KAA6K,CAAC,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,uQAAkQ,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,qFAAqF,CAAC,EAAeF,EAAE,KAAK,CAAC,SAAS,CAAcE,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,gCAAgC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,kCAAkC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,4CAA4C,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,qPAA2O,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,8WAA8W,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,6RAAwR,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,iRAAuQ,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,sKAAmLE,EAAE,SAAS,CAAC,SAAS,qEAAqE,CAAC,EAAE,GAAG,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,oGAAoG,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,iDAAiD,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,kMAAkM,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,2CAA2C,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,sDAAmEE,EAAEC,EAAE,CAAC,KAAK,qFAAqF,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,kBAAkB,CAAC,CAAC,CAAC,EAAE,yMAAyM,CAAC,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,8IAA8I,CAAC,EAAeA,EAAE,MAAM,CAAC,IAAI,+CAA+C,UAAU,eAAe,OAAO,MAAM,IAAI,sEAAsE,OAAO,iWAAiW,MAAM,CAAC,YAAY,aAAa,EAAE,MAAM,MAAM,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,gDAAgD,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,gTAAgT,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,mSAAyR,CAAC,EAAeA,EAAE,MAAM,CAAC,IAAI,uFAAuF,UAAU,eAAe,OAAO,MAAM,IAAI,uEAAuE,OAAO,qWAAqW,MAAM,CAAC,YAAY,aAAa,EAAE,MAAM,MAAM,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,sCAAsC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,0HAA0H,CAAC,EAAeA,EAAE,MAAM,CAAC,IAAI,kHAAkH,UAAU,eAAe,OAAO,MAAM,IAAI,uEAAuE,OAAO,qWAAqW,MAAM,CAAC,YAAY,aAAa,EAAE,MAAM,MAAM,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,iHAAiH,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,+KAA+K,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,yCAAyC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,2HAA2H,CAAC,EAAeA,EAAE,MAAM,CAAC,IAAI,kHAAkH,UAAU,eAAe,OAAO,MAAM,IAAI,sEAAsE,OAAO,iWAAiW,MAAM,CAAC,YAAY,YAAY,EAAE,MAAM,MAAM,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,iHAAiH,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,gIAAgI,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,4CAA4C,CAAC,EAAeF,EAAE,KAAK,CAAC,SAAS,CAAcE,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,yEAAyE,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,iHAAiH,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,0LAAqL,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,qCAAqC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,+WAAqW,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,mZAAyY,CAAC,EAAeA,EAAE,MAAM,CAAC,IAAI,iDAAiD,UAAU,eAAe,OAAO,OAAO,IAAI,uEAAuE,OAAO,mcAAmc,MAAM,CAAC,YAAY,aAAa,EAAE,MAAM,MAAM,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,gDAAgD,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,uKAAuK,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,wDAAwD,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,4JAAuJ,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,kBAAkB,CAAC,EAAE,gVAAmVA,EAAEC,EAAE,CAAC,KAAK,4CAA4C,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,cAAc,CAAC,CAAC,CAAC,EAAE,yCAAyC,CAAC,CAAC,EAAeJ,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,qDAAqD,CAAC,EAAE,oJAAoJ,CAAC,CAAC,EAAeF,EAAE,KAAK,CAAC,MAAM,CAAC,qBAAqB,OAAO,0BAA0B,QAAQ,sBAAsB,kBAAkB,6BAA6B,MAAM,0BAA0B,MAAM,EAAE,SAAS,CAAcE,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,mCAAmC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,0CAA0C,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,yCAAyC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,mDAAmD,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,4BAA4B,CAAC,EAAE,8JAAyJ,CAAC,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,6BAA6B,CAAC,EAAE,kOAAkO,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,mUAA8T,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAEC,EAAE,CAAC,KAAK,gCAAgC,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,6BAA6B,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeM,EAAuBV,EAAIC,EAAS,CAAC,SAAS,CAAcC,EAAE,IAAI,CAAC,SAAS,wQAAwQ,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,8KAA8K,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,+BAA+B,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,qYAAqY,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,0bAA0b,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,weAAwe,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,gDAAgD,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,oHAAoH,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,gDAAgD,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,mZAAmZ,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,+BAA+B,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,qcAAqc,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,oBAAoB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,maAAma,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,wCAAwC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,oeAAoe,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,kCAAkC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,glBAAglB,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,+BAA+B,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,4YAA4Y,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,oNAAoN,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,6BAA6B,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,2KAA2K,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,mBAAmB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,6CAA6C,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,SAAS,CAAC,SAAS,mBAAmB,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,0QAA0Q,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,mSAAmS,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,6VAA6V,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,SAAS,CAAC,SAAS,sBAAsB,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,mTAAmT,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,6cAA6c,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,2JAA2J,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,mUAAmU,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,yNAAyN,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,SAAS,CAAC,SAAS,mBAAmB,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,+UAA+U,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,wPAAwP,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,wBAAwB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,gHAAgH,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,SAAS,CAAC,SAAS,gBAAgB,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,6WAA6W,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,+QAA+Q,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,SAAS,CAAC,SAAS,iBAAiB,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,oTAAoT,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,kiBAAkiB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,SAAS,CAAC,SAAS,qBAAqB,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,uNAAuN,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,iiBAAiiB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,qUAAqU,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,wPAAwP,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,SAAS,CAAC,SAAS,wBAAwB,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,4pBAA4pB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,mbAAmb,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,0eAA0e,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,gWAAgW,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,0BAA0B,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,2JAA2J,CAAC,EAAeF,EAAE,KAAK,CAAC,MAAM,CAAC,qBAAqB,OAAO,0BAA0B,QAAQ,sBAAsB,kBAAkB,6BAA6B,MAAM,0BAA0B,MAAM,EAAE,SAAS,CAAcE,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,6LAA6L,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,6NAA6N,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,iRAAiR,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,sWAAsW,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,yMAAyM,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,wDAAwD,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,8QAA8Q,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,2OAA2O,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,oFAAoF,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,4YAA4Y,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,gTAAgT,CAAC,EAAeF,EAAE,KAAK,CAAC,MAAM,CAAC,qBAAqB,OAAO,0BAA0B,QAAQ,sBAAsB,kBAAkB,6BAA6B,MAAM,0BAA0B,MAAM,EAAE,SAAS,CAAcE,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,oUAAoU,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,uPAAuP,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,0WAA0W,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,mYAAmY,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,+TAA+T,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,4aAA4a,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,mOAAmO,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,6NAA6N,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,sCAAsC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,mHAAmH,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,SAAS,CAAC,SAAS,6BAA6B,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,ygBAAygB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,ocAAoc,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,SAAS,CAAC,SAAS,6BAA6B,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,gQAAgQ,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,gOAAgO,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,gTAAgT,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,6CAA6C,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,0VAA0V,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,sbAAsb,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,mRAAmR,CAAC,CAAC,CAAC,CAAC,EACx8pIS,EAAqB,CAAC,QAAU,CAAC,UAAY,CAAC,KAAO,WAAW,YAAc,CAAC,sBAAwB,GAAG,CAAC,EAAE,UAAY,CAAC,KAAO,WAAW,YAAc,CAAC,sBAAwB,GAAG,CAAC,EAAE,UAAY,CAAC,KAAO,WAAW,YAAc,CAAC,sBAAwB,GAAG,CAAC,EAAE,UAAY,CAAC,KAAO,WAAW,YAAc,CAAC,sBAAwB,GAAG,CAAC,EAAE,SAAW,CAAC,KAAO,WAAW,YAAc,CAAC,sBAAwB,GAAG,CAAC,EAAE,UAAY,CAAC,KAAO,WAAW,YAAc,CAAC,sBAAwB,GAAG,CAAC,EAAE,UAAY,CAAC,KAAO,WAAW,YAAc,CAAC,sBAAwB,GAAG,CAAC,EAAE,mBAAqB,CAAC,KAAO,UAAU,CAAC,CAAC",
  "names": ["richText", "u", "x", "p", "Link", "motion", "richText1", "richText2", "richText3", "richText4", "richText5", "richText6", "__FramerMetadata__"]
}