{
  "version": 3,
  "sources": ["ssg:https://framerusercontent.com/modules/YVgynkVt96a0ES9Gk8yz/e2OiwG21hMIpRxQ1FbTF/eo4RAmtig-25.js"],
  "sourcesContent": ["import{jsx as e,jsxs as t}from\"react/jsx-runtime\";import{Link as n}from\"framer\";import{motion as a}from\"framer-motion\";import*as i from\"react\";export const richText=/*#__PURE__*/t(i.Fragment,{children:[/*#__PURE__*/e(\"p\",{children:\"Prompt engineering has been around for a while, but the term has gone viral with the recent boom in large language models (LLMs). The main driver of the phenomenon is that zero-shot and few-shot learning work well only with language models of a very large size. \"}),/*#__PURE__*/t(\"p\",{children:[\"The hype around prompt engineering was inevitable \u2014 after all, prompt wording can have a dramatic effect on the quality of LLM output (\",/*#__PURE__*/e(n,{href:\"https://arxiv.org/pdf/2102.09690.pdf\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,relValues:[],scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(a.a,{children:\"LLMs exhibit large variance over different prompt templates\"})}),\"). It is also an effective way to downstream language models without retraining any model parameters or gathering data required for traditional fine-tuning. \"]}),/*#__PURE__*/e(\"img\",{alt:\"It is also an effective way to downstream language models without retraining any model parameters or gathering data required for traditional fine-tuning\",className:\"framer-image\",height:\"856\",src:\"https://framerusercontent.com/images/fXTVKsUG7gKlHDH9oISktv8sS8.webp\",srcSet:\"https://framerusercontent.com/images/fXTVKsUG7gKlHDH9oISktv8sS8.webp?scale-down-to=512 512w,https://framerusercontent.com/images/fXTVKsUG7gKlHDH9oISktv8sS8.webp?scale-down-to=1024 1024w,https://framerusercontent.com/images/fXTVKsUG7gKlHDH9oISktv8sS8.webp?scale-down-to=2048 2048w,https://framerusercontent.com/images/fXTVKsUG7gKlHDH9oISktv8sS8.webp 2560w\",style:{aspectRatio:\"2560 / 1712\"},width:\"1280\"}),/*#__PURE__*/e(\"p\",{children:\"The research literature suggests a variety of promising prompt engineering techniques. But we have yet to discover a universal \u201Cfind the best prompt for a certain task\u201D algorithm suitable for every LLM. We have to experiment with each specific model and task to find a prompt that produces the desired output. That\u2019s why the \u201CPrompt Engineer\u201D job became a thing.\"}),/*#__PURE__*/e(\"h2\",{children:\"Prompt Engineers\"}),/*#__PURE__*/e(\"img\",{alt:\"Prompt Engineers\",className:\"framer-image\",height:\"751\",src:\"https://framerusercontent.com/images/5Wun6gU6Ur6Ss4kVmjRHApw5oJA.webp\",srcSet:\"https://framerusercontent.com/images/5Wun6gU6Ur6Ss4kVmjRHApw5oJA.webp?scale-down-to=512 512w,https://framerusercontent.com/images/5Wun6gU6Ur6Ss4kVmjRHApw5oJA.webp?scale-down-to=1024 1024w,https://framerusercontent.com/images/5Wun6gU6Ur6Ss4kVmjRHApw5oJA.webp?scale-down-to=2048 2048w,https://framerusercontent.com/images/5Wun6gU6Ur6Ss4kVmjRHApw5oJA.webp 2560w\",style:{aspectRatio:\"2560 / 1502\"},width:\"1280\"}),/*#__PURE__*/t(\"p\",{children:[\"As it happens with every hype position in IT (speaking from my personal work experience as a Developer Advocate), there is a lot of confusion for both employers and employees about what a Prompt Engineer has to know. The confusion is made worse by the common perception that prompt engineering is a process of reformulating questions to an oracle until you get a desired answer, something like a patient parent doing math homework with their kid. On the contrary, prompt engineering is a very deliberate procedure. Even more importantly, \u201C\",/*#__PURE__*/e(n,{href:\"https://www.promptingguide.ai/\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,relValues:[],scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(a.a,{children:\"prompt engineering is not just about designing and developing prompts. It encompasses a wide range of skills and techniques that are useful for interacting and developing with LLMs\"})}),\"\u201D.\"]}),/*#__PURE__*/t(\"p\",{children:[\"There are plenty of helpful \",/*#__PURE__*/e(n,{href:\"https://www.promptingguide.ai/\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,relValues:[],scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(a.a,{children:\"guides\"})}),\", \",/*#__PURE__*/e(n,{href:\"https://learn.deeplearning.ai/login?redirect_course=chatgpt-prompt-eng\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,relValues:[],scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(a.a,{children:\"courses\"})}),\" and \",/*#__PURE__*/e(n,{href:\"https://towardsdatascience.com/what-i-learned-pushing-prompt-engineering-to-the-limit-c40f0740641f\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,relValues:[],scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(a.a,{children:\"articles\"})}),\" with techniques and best practices of prompt engineering which I used myself for getting the hang of it. And in most of them, based on my experience, you will see the same recommendation: good prompts are born out of experimentation, so you need to establish your own process of designing a prompt and develop a sixth sense for prompt engineering. Obviously, some of the intuition can come from working experience as a Machine Learning Engineer, which helps develop a deeper understanding of how LLMs are built and trained. But there are some less obvious professions that can naturally develop strong intuition in prompt engineering. One of them is a Crowd Solution Architect (CSA).\"]}),/*#__PURE__*/e(\"h2\",{children:\"Crowd Solution Architects\"}),/*#__PURE__*/e(\"img\",{alt:\"Crowd Solution Architects\",className:\"framer-image\",height:\"707\",src:\"https://framerusercontent.com/images/58MwBtf9NWeAocWTSWctrHtbk4.webp\",srcSet:\"https://framerusercontent.com/images/58MwBtf9NWeAocWTSWctrHtbk4.webp?scale-down-to=512 512w,https://framerusercontent.com/images/58MwBtf9NWeAocWTSWctrHtbk4.webp?scale-down-to=1024 1024w,https://framerusercontent.com/images/58MwBtf9NWeAocWTSWctrHtbk4.webp?scale-down-to=2048 2048w,https://framerusercontent.com/images/58MwBtf9NWeAocWTSWctrHtbk4.webp 2560w\",style:{aspectRatio:\"2560 / 1414\"},width:\"1280\"}),/*#__PURE__*/e(\"p\",{children:\"It might seem like I\u2019m replacing one trendy, misunderstood IT job title with another one that is just as baffling and newfangled. Well, it\u2019s actually not that new \u2014 Crowdsourcing Solution Architects have been around since the emergence of crowdsourcing platforms, so CSAs can easily have 15+ years of experience. The industry already has a solid understanding of the professional scope of a CSA.\"}),/*#__PURE__*/e(\"p\",{children:\"So, what are their day-to-day tasks, and why can their experience be helpful for developing intuition in prompt engineering? To put it simply, CSAs design data labeling tasks for a crowd (expert or non-expert), converting initial specifications to a format that is easy for labelers to understand. This includes writing detailed instructions, creating a user-friendly labeling interface, setting up a fair pricing scheme and quality-control mechanisms, and, most importantly, applying sophisticated decomposition techniques to an initial problem. Decomposition is an important skill that turns a hard task into a set of easier subtasks. An example is to transform a ranking problem into a side-by-side labeling task followed by noisy Bradley-Terry aggregation. The subtasks are solved by the crowd and then CSAs apply various aggregation techniques to get the final answer to the problem.\"}),/*#__PURE__*/e(\"img\",{alt:\"The subtasks are solved by the crowd and then CSAs apply various aggregation techniques to get the final answer to the problem\",className:\"framer-image\",height:\"708\",src:\"https://framerusercontent.com/images/X0EQRoH2E2yJG8OLTlQ61n2DDE.webp\",srcSet:\"https://framerusercontent.com/images/X0EQRoH2E2yJG8OLTlQ61n2DDE.webp?scale-down-to=512 512w,https://framerusercontent.com/images/X0EQRoH2E2yJG8OLTlQ61n2DDE.webp?scale-down-to=1024 1024w,https://framerusercontent.com/images/X0EQRoH2E2yJG8OLTlQ61n2DDE.webp?scale-down-to=2048 2048w,https://framerusercontent.com/images/X0EQRoH2E2yJG8OLTlQ61n2DDE.webp 2560w\",style:{aspectRatio:\"2560 / 1417\"},width:\"1280\"}),/*#__PURE__*/e(\"h2\",{children:\"Why does designing crowdsourcing tasks help develop intuition in prompt engineering\"}),/*#__PURE__*/e(\"img\",{alt:\"Why does designing crowdsourcing tasks help develop intuition in prompt engineering\",className:\"framer-image\",height:\"1714\",src:\"https://framerusercontent.com/images/MhVuf0HAU74GbmKdQ17tBa271TA.webp\",srcSet:\"https://framerusercontent.com/images/MhVuf0HAU74GbmKdQ17tBa271TA.webp?scale-down-to=1024 764w,https://framerusercontent.com/images/MhVuf0HAU74GbmKdQ17tBa271TA.webp?scale-down-to=2048 1529w,https://framerusercontent.com/images/MhVuf0HAU74GbmKdQ17tBa271TA.webp 2560w\",style:{aspectRatio:\"2560 / 3428\"},width:\"1280\"}),/*#__PURE__*/e(\"p\",{children:\"While experimenting with prompt engineering, I noticed similarities between best practices in designing crowdsourcing tasks and best practices in prompt engineering. This led me to develop and test several hypotheses on how CSA methods could be used for improving prompt results. I strongly believe that new prompting techniques may be discovered by applying knowledge from adjacent areas of research, such as crowdsourcing.\"}),/*#__PURE__*/e(\"h2\",{children:\"Decomposition + Aggregation\"}),/*#__PURE__*/t(\"p\",{children:[\"Techniques like \",/*#__PURE__*/e(n,{href:\"https://www.promptingguide.ai/techniques/cot\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,relValues:[],scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(a.a,{children:\"chain-of-thought prompting\"})}),\" and \",/*#__PURE__*/e(n,{href:\"https://arxiv.org/pdf/2205.10625.pdf\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,relValues:[],scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(a.a,{children:\"least-to-most prompting\"})}),\" demonstrate that a model performs better if a task is divided into subtasks or steps. Decomposition works the same way for crowdsourcing tasks by breaking them down into smaller problems that are much easier to solve. It is intuitively easy to understand why such a technique helps to increase human labeling quality \u2014 it makes it easier to concentrate and requires less expertise from labelers. For LLMs it might work well because the subtasks might occur more frequently in LLM training datasets scraped from the web.\"]}),/*#__PURE__*/t(\"p\",{children:[\"CSAs have learned in practice how to decompose tasks for \",/*#__PURE__*/e(n,{href:\"https://towardsdatascience.com/evaluating-search-relevance-on-demand-with-crowdsourcing-4203b7058101\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,relValues:[],scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(a.a,{children:\"search relevance estimation\"})}),\", \",/*#__PURE__*/e(n,{href:\"https://youtu.be/PyZI-O9KcC4?t=653\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,relValues:[],scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(a.a,{children:\"3D transport labeling\"})}),\", \",/*#__PURE__*/e(n,{href:\"https://www.youtube.com/watch?v=NUXVNlKg5PM\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,relValues:[],scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(a.a,{children:\"ad moderation\"})}),\", \",/*#__PURE__*/e(n,{href:\"https://thesequence.substack.com/p/guest-post-how-to-build-a-responsible\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,relValues:[],scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(a.a,{children:\"personal information recognition in code\"})}),\", and many other use cases. They have developed a strong intuition for what is hard and what is easy to solve with a crowd. This intuition can be partially captured in a set of best practices. For example, there is a rule of thumb that the number of classes in classification tasks shouldn\u2019t exceed 5 or 6, otherwise the choice overload bias comes into play. Another rule is that class elimination one-by-one from simple to complex will provide better results than labeling classes at the same time. Can these rules be applied for labeling with LLMs? It seems so! We applied the latter rule to labeling with GPT-4 and the results showed 20% higher accuracy.\"]}),/*#__PURE__*/t(\"p\",{children:[\"When there is a decomposition, there is also an aggregation. For example, when decomposing ranking tasks to side-by-side labeling, one needs to aggregate them with the \",/*#__PURE__*/e(n,{href:\"https://toloka.ai/docs/crowd-kit/reference/crowdkit.aggregation.pairwise.bradley_terry.BradleyTerry/\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,relValues:[],scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(a.a,{children:\"Bradley-Terry\"})}),\" (or any other suitable) model. The closest thing related to aggregation in prompt engineering is, from my perspective, \",/*#__PURE__*/e(n,{href:\"https://arxiv.org/pdf/2203.11171.pdf\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,relValues:[],scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(a.a,{children:\"consistent chain-of-thought prompting\"})}),\": it is based on a \",/*#__PURE__*/e(n,{href:\"https://toloka.ai/docs/crowd-kit/reference/crowdkit.aggregation.classification.majority_vote.MajorityVote/#:~:text=The%20Majority%20Vote%20aggregation%20algorithm,are%20assigned%20to%20workers'%20votes.\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,relValues:[],scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(a.a,{children:\"Majority Vote Aggregation algorithm\"})}),\". It seems like it might be a very interesting research problem to test more sophisticated aggregation techniques in labeling with LLMs.\"]}),/*#__PURE__*/e(\"h2\",{children:\"Instructions\"}),/*#__PURE__*/t(\"p\",{children:[\"One of the keys to getting good results with crowdsourcing is to provide clear and detailed instructions (basically, a detailed prompt for humans). Since LLMs are incapable of reasoning, giving them the perfect instructions for people might not lead to the best output quality, as shown in the article \",/*#__PURE__*/e(n,{href:\"https://arxiv.org/pdf/2010.11982.pdf\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,relValues:[],scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(a.a,{children:\"The Turking Test: Can Language Models Understand Instructions?\"})}),\".\"]}),/*#__PURE__*/e(\"p\",{children:\"However, some best practices for prompt engineering and instruction design for crowdsourcing match one-to-one, proving that a skilled CSA could make a promising prompt engineer. Here are some practices that apply to prompts and instructions:\"}),/*#__PURE__*/t(\"ul\",{style:{\"--framer-font-size\":\"18px\",\"--framer-text-alignment\":\"start\",\"--framer-text-color\":\"rgb(30, 33, 38)\",\"--framer-text-stroke-width\":\"0px\",\"--framer-text-transform\":\"none\"},children:[/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Short does not equal good. Be as clear as possible.\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(n,{href:\"https://arxiv.org/pdf/2005.14165.pdf\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,relValues:[],scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(a.a,{children:\"Since LLMs are few shot learners like we are\"})}),\", examples matter a lot.\"]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"If there are multiple possible classes in the task, provide enough examples (at least 2\u20133) to illustrate each of them.\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Use real-life (not synthetic) examples for your few-shots.\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"If you do add rare cases to the task, make sure to explain them well in the instructions.\"})})]}),/*#__PURE__*/e(\"h2\",{children:\"Key takeaways\"}),/*#__PURE__*/t(\"ul\",{style:{\"--framer-font-size\":\"18px\",\"--framer-text-alignment\":\"start\",\"--framer-text-color\":\"rgb(30, 33, 38)\",\"--framer-text-stroke-width\":\"0px\",\"--framer-text-transform\":\"none\"},children:[/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"There is a promising unexplored research field for applying CSA techniques to prompt engineering.\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Intuition in prompt engineering certainly could be gained through other work experience.\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"At Toloka, we successfully use our CSAs as prompt engineers. If you happen to have someone on your team who worked with crowdsourcing, you don\u2019t have to search for a prompt engineer! \"})})]})]});export const richText1=/*#__PURE__*/t(i.Fragment,{children:[/*#__PURE__*/e(\"h2\",{children:\"Challenges\"}),/*#__PURE__*/t(\"p\",{children:[\"A dating platform faced a significant challenge in moderating user-generated content for \",/*#__PURE__*/e(\"strong\",{children:\"scam\"}),\", \",/*#__PURE__*/e(\"strong\",{children:\"abuse\"}),\", and \",/*#__PURE__*/e(\"strong\",{children:\"threat\"}),\" context. \"]}),/*#__PURE__*/e(\"p\",{children:\"User-generated context comprised of\"}),/*#__PURE__*/t(\"ol\",{children:[/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"User profiles with images and text captions\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Chats messages\"})})]}),/*#__PURE__*/e(\"p\",{children:\"Before client came to Toloka the platform relied on\"}),/*#__PURE__*/t(\"ol\",{children:[/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Manual human moderation, which was time-consuming and unable to cover spikes, not always effective and citing client \u201Cputting unnecessary burden of watching cruel content daily\u201D\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Simple keywords lists and filters that did not cover more complex cases and contexts \"})})]}),/*#__PURE__*/e(\"p\",{children:\"When the platform reached a whole new level of MAU their community started being more and more toxic and the metrics were falling down.\"}),/*#__PURE__*/e(\"p\",{children:\"The platform needed a scalable, ready-to-go and accurate solution to moderate images and texts in real-time, while also improving the accuracy of the moderation model over time.\"}),/*#__PURE__*/e(\"h2\",{children:\"Solution\"}),/*#__PURE__*/e(\"p\",{children:\"To overcome these challenges, the dating platform implemented Toloka moderation service \u2014 a machine learning-based solution for image and text moderation. The solution used Toloka a combination of natural language processing (NLP) and computer vision (CV) algorithms to detect and tag inappropriate images and texts.\"}),/*#__PURE__*/e(\"p\",{children:\"Toloka models captured from the scratch scam, abuse and threat as pre-defined classes. We were continuously improving model accuracy with human insight of Toloka crowd labelling dataset of user profiles and captions. This helped to identify patterns and contexts that the model may not have detected, and to continuously improve the model's accuracy.\"}),/*#__PURE__*/e(\"p\",{children:\"Moreover, the client was able to add phrases specific to their particular community on the go using platform interface.\"}),/*#__PURE__*/e(\"p\",{children:\"For personal chats Toloka moderation model analysed not just each message but a bunch of messages to keep the whole context of message.\"}),/*#__PURE__*/e(\"h2\",{children:\"Business Impact Metrics\"}),/*#__PURE__*/e(\"p\",{children:\"The implementation of the machine learning-based moderation solution had a significant impact on the dating platform's business metrics.\"}),/*#__PURE__*/t(\"ol\",{children:[/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"The service level agreement (SLA) for detecting scammers was reduced by 85%, resulting in a safer and more trusted platform for users.\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"The platform's premium subscription churn rate decreased by 12% after 6 months of successful partnership with Toloka, and the lifetime value of overall users increased by 8%.\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"The client cut costs on moderation by 53% per one item. \"})})]}),/*#__PURE__*/e(\"p\",{children:\"Overall, the implementation of machine learning for image and text moderation with human insight was a successful solution to the challenges faced by the dating platform. The platform was able to provide a safer and more secure user experience, while also increasing user engagement and revenue.\"})]});export const richText2=/*#__PURE__*/t(i.Fragment,{children:[/*#__PURE__*/e(\"h2\",{children:\"About the client\"}),/*#__PURE__*/e(\"p\",{children:\"Any online marketplace relies on their product search to help customers find items and drive a large portion of sales. Our client, one of the largest e-commerce platforms in the EMEA region, aimed to boost their revenue by improving the quality of search results. \"}),/*#__PURE__*/e(\"h2\",{children:\"Challenge\"}),/*#__PURE__*/e(\"p\",{children:\"The client\u2019s team knew that their site search was underperforming, but they didn\u2019t know exactly why. They chose the most efficient path to discover the underlying problems \u2014 offline evaluation of search results.\"}),/*#__PURE__*/e(\"p\",{children:\"The goal was to perform dissatisfaction analytics (DSAT), which involves in-depth analysis of specific cases where a machine learning model or product fails to provide an acceptable level of service. DSAT is a high-precision tool that can help teams identify pain points and enhance their product.\"}),/*#__PURE__*/e(\"h2\",{children:\"Solution\"}),/*#__PURE__*/e(\"p\",{children:\"The Toloka team set up a DSAT process to identify problems in the client\u2019s product search.\"}),/*#__PURE__*/e(\"p\",{children:\"The overall process has 5 steps:\"}),/*#__PURE__*/t(\"ol\",{style:{\"--framer-font-size\":\"18px\",\"--framer-text-alignment\":\"start\",\"--framer-text-color\":\"rgb(30, 33, 38)\",\"--framer-text-stroke-width\":\"0px\",\"--framer-text-transform\":\"none\",\"--list-style-type\":\"unset\"},children:[/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Select a stratified sample of search queries to analyze for search relevance.\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Label relevance for pairs of queries and search results.\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Extract results labeled as \u201Cirrelevant\u201D and categorize them to find the queries that represent fixable problems or pain points.\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Label the queries with fixable problems and identify which issues are most prevalent.\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Fix issues in search algorithms and measure search quality to track improvements.\"})})]}),/*#__PURE__*/e(\"p\",{children:\"Ideally, these steps are repeated on a regular basis (every quarter, for example).\"}),/*#__PURE__*/e(\"p\",{children:\"Let\u2019s look at how we handled each step.\"}),/*#__PURE__*/e(\"h3\",{children:\"Step 1. Sampling search queries\"}),/*#__PURE__*/e(\"p\",{children:\"The client provided data on search queries and we selected 20,000 random queries based on their frequency. The resulting dataset contained a balanced set of queries with high frequency, average frequency, and low frequency.\"}),/*#__PURE__*/e(\"h3\",{children:\"Step 2. Labeling search relevance\"}),/*#__PURE__*/e(\"p\",{children:\"Once we acquired our sample of search queries, we paired them with search results and labeled search relevance using Toloka\u2019s global crowd. To guarantee labeling accuracy, we used high overlap, meaning multiple people rated each query and the results were aggregated.\"}),/*#__PURE__*/e(\"h3\",{children:\"Step 3. Categorizing \u201Cirrelevant\u201D results\"}),/*#__PURE__*/e(\"p\",{children:\"About 5% of the search queries showed no relevant products in the top six search results. We focused on this set (about 1000 queries) and asked the crowd to categorize them using a series of questions. This helped weed out pointless search sessions, like nonsense and products that aren\u2019t sold on the site. The result was a clearly defined set of queries where the product search should have shown relevant items, but failed.\"}),/*#__PURE__*/e(\"p\",{children:\"The image shows the overall process and the questions used for categorizing results.\"}),/*#__PURE__*/e(\"img\",{alt:\"Overview of the DSAT process and questions for categorizing failed searches\",className:\"framer-image\",height:\"1540\",src:\"https://framerusercontent.com/images/5xVgtmLHIxLWBV9YL4Us7JdFZmc.jpeg\",srcSet:\"https://framerusercontent.com/images/5xVgtmLHIxLWBV9YL4Us7JdFZmc.jpeg?scale-down-to=512 512w,https://framerusercontent.com/images/5xVgtmLHIxLWBV9YL4Us7JdFZmc.jpeg?scale-down-to=1024 1024w,https://framerusercontent.com/images/5xVgtmLHIxLWBV9YL4Us7JdFZmc.jpeg?scale-down-to=2048 2048w,https://framerusercontent.com/images/5xVgtmLHIxLWBV9YL4Us7JdFZmc.jpeg?scale-down-to=4096 4096w,https://framerusercontent.com/images/5xVgtmLHIxLWBV9YL4Us7JdFZmc.jpeg 5630w\",style:{aspectRatio:\"5630 / 3080\"},width:\"2815\"}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"Overview of the DSAT process and questions for categorizing failed searches.\"})}),/*#__PURE__*/e(\"h3\",{children:\"Step 4. Identifying issues\"}),/*#__PURE__*/e(\"p\",{children:\"Each \u201Cproblematic\u201D query in the final set was labeled to identify the problem. Three main issues were discovered:\"}),/*#__PURE__*/t(\"ul\",{style:{\"--framer-font-size\":\"18px\",\"--framer-text-alignment\":\"start\",\"--framer-text-color\":\"rgb(30, 33, 38)\",\"--framer-text-stroke-width\":\"0px\",\"--framer-text-transform\":\"none\"},children:[/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Wrong category: search results were shown for the wrong category of products (like books instead of electronics).\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Wrong sorting: search results were sorted incorrectly, with irrelevant items at the top.\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Typos: misspelled words in the query were not detected and the intent was misunderstood.\"})})]}),/*#__PURE__*/e(\"p\",{children:\"After we identified the percentage of failed searches affected by each type of issue, the team was able to prioritize which issues to tackle first.\"}),/*#__PURE__*/e(\"img\",{alt:\"Main issues discovered and their prevalence, used for prioritizing improvements\",className:\"framer-image\",height:\"734\",src:\"https://framerusercontent.com/images/PqdBVotlkuvFPHaIjZGwY4WPI.jpeg\",srcSet:\"https://framerusercontent.com/images/PqdBVotlkuvFPHaIjZGwY4WPI.jpeg?scale-down-to=512 512w,https://framerusercontent.com/images/PqdBVotlkuvFPHaIjZGwY4WPI.jpeg?scale-down-to=1024 1024w,https://framerusercontent.com/images/PqdBVotlkuvFPHaIjZGwY4WPI.jpeg?scale-down-to=2048 2048w,https://framerusercontent.com/images/PqdBVotlkuvFPHaIjZGwY4WPI.jpeg 2474w\",style:{aspectRatio:\"2474 / 1468\"},width:\"1237\"}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"Main issues discovered and their prevalence, used for prioritizing improvements.\"})}),/*#__PURE__*/e(\"h2\",{children:\"Business impact\"}),/*#__PURE__*/e(\"p\",{children:\"Offline evaluation with DSAT helped to pinpoint three main issues to focus on for improving product search. The team then used search quality metrics in an improvement cycle to measure the impact of changes in the target areas.\"}),/*#__PURE__*/e(\"p\",{children:\"The result was 8% better search relevancy overall, with a clear connection to GMV growth for the marketplace.\"})]});export const richText3=/*#__PURE__*/t(i.Fragment,{children:[/*#__PURE__*/e(\"p\",{children:\"The spread of AI has already driven significant technological advances and increased operational efficiency. Among the notable applications of AI is the generation of top-quality pictures, photographs, concept art, music, and various texts, including abstracts and fiction, as well as videos and animations.\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI possesses such capabilities. It represents a creative tool that produces something that has never existed before. It is likely to fundamentally change the processes associated with art and creativity as well as business processes. Below we take a closer look at what generative AI is and why it has recently gained so much public attention.\"}),/*#__PURE__*/e(\"h2\",{children:\"The difference between Traditional and Generative AI\"}),/*#__PURE__*/e(\"p\",{children:\"The fundamental difference between traditional and generative AI lies in their capabilities and applications.\"}),/*#__PURE__*/e(\"p\",{children:\"Traditional AI systems are mainly employed in data analysis and making predictions. It is referred to as narrow AI, which is tailored to process a particular set of input data. Such systems are capable of learning from the data and making decisions or predictions based on it. It is important to note that they require labeled training data.\"}),/*#__PURE__*/e(\"p\",{children:\"Traditional AI tools are capable of making decisions within the sets of rules they have been taught. Voice assistants, search algorithms, and recommendation systems are some common applications of traditional AI. Such traditional AI algorithms are quite effective and help make life easier, although they cannot create something new, which is not the case with generative AI. This type of artificial intelligence differs from traditional systems in the fact that it creates completely new content.\"}),/*#__PURE__*/e(\"h2\",{children:\"What is generative AI?\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI has been rapidly developing over the past few years. Therefore, you can think of it as the next generation of artificial intelligence. It represents a form of AI technologies that are empowered to create new content.\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI creates new content such as realistic images, photos, pictures, texts, videos, etc. by mimicking already existing pieces of content.\"}),/*#__PURE__*/e(\"p\",{children:\"As mentioned above, unlike traditional AI, generative AI creates something entirely novel out of the information it has been given. It can create original, creative content, be it text, images, music, or computer code. Generative AI models are trained on a huge set of data and, by picking up the underlying patterns, produce fresh pieces of content that replicate, but don't exactly copy, the training data set. That is, they generate unique data as if it were created by a human.\"}),/*#__PURE__*/e(\"p\",{children:\"Popular generative AI interfaces involve, for instance, one of the large language models GPT-4, which employs a neural network trained on gigantic sets of existing data from the Internet. It is designed for natural language processing.\"}),/*#__PURE__*/e(\"p\",{children:\"DALL-E, MidJourney, and Stable Diffusion are machine-learning models that can create images based on text prompts.\"}),/*#__PURE__*/e(\"h2\",{children:\"Using generative AI\"}),/*#__PURE__*/e(\"p\",{children:\"Turning raw data into an entirely new experience that can be applied across a wide range of fields, from automating routine tasks to creating pieces of art. By producing fresh content, generative AI serves as an aid but not a replacement for writers, graphic designers, visual artists, and musicians, as well as other professions.\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI is an immensely influential and powerful instrument that can be employed to generate aesthetically appealing and very convincing content. Some of its typical applications by content type and field of use include:\"}),/*#__PURE__*/e(\"h3\",{children:\"Text\"}),/*#__PURE__*/e(\"p\",{children:\"Generative models are capable of easily producing unique texts on a variety of topics. In business, such tools can assist with composing marketing collateral, reports,business documentation, creating articles drafts, and preparing presentations.\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI can transform natural language into code, and also fix bugs. It can help developers reduce the time needed for software development, at the same time improve the quality of code and simplify the development process.\"}),/*#__PURE__*/e(\"h3\",{children:\"Images\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI transforms textual prompts into life-like images, building new scenes and modeling never-before-seen artwork. For instance, some AI-generated graphics might be almost undetectable from live photo realistic images.\"}),/*#__PURE__*/e(\"h3\",{children:\"Music\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI models may create authentic music with AI algorithms generating original tunes and beats. The system creates a new piece of composition by analyzing existing music.\"}),/*#__PURE__*/e(\"h3\",{children:\"Video\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI produces high-quality videos resembling the real ones. In addition to the automated production of video from text, generative AI tools can assemble short videos based on available images.\"}),/*#__PURE__*/e(\"h3\",{children:\"Marketing\"}),/*#__PURE__*/e(\"p\",{children:\"Advertising companies utilize generative AI models to quickly create unique promotional materials, which include AI-generated images and texts. It helps marketing teams to facilitate their work and accomplish projects faster. AI can generate dozens or hundreds of unique and eye-catching photos and illustrations, imaginative product descriptions, articles, and other promotional content in a short time. Thus, machine learning models save the company's resources on the manual creation of original content.\"}),/*#__PURE__*/e(\"h3\",{children:\"Design\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI helps marketing specialists and designers to create multiple new possible product design options based on past designs. Also, AI tools may provide inspiration for creating illustrations based on the content suggested by the system.\"}),/*#__PURE__*/e(\"h2\",{children:\"How Does Generative AI Work?\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI is a form of machine learning that, at its essence, combines deep learning neural network algorithms and techniques to create content based on patterns it observes in existing content. A huge amount of available content is uploaded to generative AI models in order to train these models to generate novel pieces of data. They learn to recognize fundamental patterns in a dataset and, when prompted, create content based on those patterns.\"}),/*#__PURE__*/e(\"p\",{children:\"The output of generative AI is most commonly original material, as mentioned earlier. However, generative AI models utilize machine learning to create content that mimics previously man-made creative work. Generative AI technologies utilize huge repositories of data and use this collected information to generate content. Many criticize AI for being able to replace humans in this way, but AI algorithms are more about aiding people, not replacing them.\"}),/*#__PURE__*/e(\"h2\",{children:\"Foundation models in generative AI\"}),/*#__PURE__*/e(\"p\",{children:\"Foundation models represent powerful artificial intelligence-trained models that employ tremendous data sets and processing power to create any kind of content, from text to visuals. Foundation models are at the core of generative AI.\"}),/*#__PURE__*/e(\"p\",{children:\"One foundation model can accomplish different goals, unlike traditional AI, where one model is trained to handle one concrete job, like detecting objects in a photo or forecast customer churn. In other words, foundation models may also perform tasks beyond content generation; they may also handle traditional AI tasks such as classification and prediction.\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI applications are commonly based on foundation models. Such models consist of artificial neural networks, which in turn consist of neurons that mimic the human brain. Layers are the underlying elements of neural networks. The more the number of layers, the higher the training capabilities of the network.\"}),/*#__PURE__*/e(\"p\",{children:\"The foundation models are categorized under such a machine learning algorithm as deep learning, since the mentioned neural networks are very extensive and include many deep layers. The foundation models that generative AI applications are based on can handle extremely vast and versatile sets of unstructured data and carry out more than one task.\"}),/*#__PURE__*/e(\"h2\",{children:\"Generative AI models\"}),/*#__PURE__*/e(\"p\",{children:\"Below are some generative AI models and the way they function:\"}),/*#__PURE__*/e(\"h3\",{children:\"Generative Adversarial Networks (GANs)\"}),/*#__PURE__*/e(\"p\",{children:\"Generative Adversarial Network (GAN) refers to a class of neural networks that create new pictures from a given dataset. GANs consist of two neural networks: a generator and a discriminator. The generator attempts to generate images that are similar to real images and the discriminator determines whether the generated content is real or not.\"}),/*#__PURE__*/e(\"p\",{children:\"GANs function by training the generator and discriminator in conjunction through what is known as a zero-sum game, where the gain of one network is treated as the loss of the other. The generator is trained to create images that best favor the discriminator. As a result, the generator learns to generate images that resemble real-life ones, allowing it to produce novel examples of pictures and expand the dataset based on existing images.\"}),/*#__PURE__*/e(\"h3\",{children:\"Variational Autoencoders (VAEs)\"}),/*#__PURE__*/e(\"p\",{children:\"Variational Autoencoder (VAE) is a deep neural network that is used to create synthetic data from existing data. VAE employs two kinds of neural networks. One network discovers the optimal ways to encode the original data into latent space, and the second network, the decoder, figures out appropriate ways to transform these latent representations into new content.\"}),/*#__PURE__*/e(\"h3\",{children:\"Transformer-based models\"}),/*#__PURE__*/e(\"p\",{children:\"Transformer is a type of neural network architecture. Transformer-based models are primarily utilized for tasks that involve data sequences, meaning information that shares particular semantics or has correlations with each other, such as natural language processing. It is perfect for determining how a few bits of information are associated with each other, for instance, how various words in a phrase are linked, or how different parts of an image match with each other.\"}),/*#__PURE__*/e(\"p\",{children:\"The transformer AI system handles the whole text at once, which means it can comprehend the entire text rather than just individual words. Therefore, it can take better account of the context of a word. This generative AI model is trained on extremely large text models and then fine-tuned for specific tasks such as translation, responding to queries, or text generation. The large language models that they produce are capable of delivering strikingly cohesive and meaningful contextualized sentences, passages, or even entire articles.\"}),/*#__PURE__*/e(\"h3\",{children:\"Diffusion models\"}),/*#__PURE__*/e(\"p\",{children:\"Diffusion models are the tools utilized to generate images from random noise. Such a model learns on a random sample of thousands of images bit by bit, where at every step a certain amount of noise is applied to the image from the sample, while the model learns to inverse this noise, thus improving the quality of the image. Thus, the trained model can generate completely novel data, gradually getting rid of the random noise on a picture by means of reverse diffusion.\"}),/*#__PURE__*/e(\"h2\",{children:\"Benefits of generative AI for business\"}),/*#__PURE__*/e(\"p\",{children:\"Generative artificial intelligence (AI) has many benefits for companies adopting it, including:\"}),/*#__PURE__*/e(\"h3\",{children:\"New ideas generation\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI is capable of suggesting new ideas, concepts, and solutions that can be applied not only in business but also in various fields such as science, technology, art, medicine, etc. For example, AI can help develop new products or services, improve manufacturing processes, or suggest new marketing strategies.\"}),/*#__PURE__*/e(\"h3\",{children:\"Improving customer experience\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI can process large amounts of customer data, which helps businesses gain a better comprehension of their clients' needs and preferences. This improves customer service and satisfaction.\"}),/*#__PURE__*/e(\"h3\",{children:\"Increased efficiency\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI is able to automate routine processes such as data processing and information analysis, which frees employees from performing these tasks. This allows them to concentrate on more complex data studies and more critical challenges, increasing overall productivity.\"}),/*#__PURE__*/e(\"h3\",{children:\"Cost reduction\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI can substitute routine manual labor and lower production costs. This is especially important for small and medium-sized organizations which are often challenged with limited resources. It can significantly reduce production costs and improve product quality.\"}),/*#__PURE__*/e(\"h3\",{children:\"Solving complex problems\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI can be helpful in solving challenging issues that could not be or were extremely difficult to be resolved. It may be utilized, for example, to discover new medications, materials, or technologies.\"}),/*#__PURE__*/e(\"h3\",{children:\"Software security\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI technology may be utilized to guard a company against cyberattacks and other security threats. It can detect and evaluate large amounts of data to detect suspicious activity and prevent attacks. Large Language Models (LLMs) may also be employed to review and analyze email contents to identify phishing or other types of security threats.\"}),/*#__PURE__*/e(\"h2\",{children:\"Why is generative AI vital today?\"}),/*#__PURE__*/e(\"p\",{children:\"Artificial intelligence technology and Generative AI in particular have numerous advantages and play an essential role in the contemporary world. This technology is the one that utilizes machine learning algorithms to generate various types of new data such as pictures, texts, music, etc. Generative AI reduces human intervention in routine tasks and allows the creation of new products, services, and ideas that can enhance the quality of our lives.\"}),/*#__PURE__*/e(\"p\",{children:\"From the ability to create unique works of art to improving customer service, generative AI provides a multitude of tools that can boost our imagination, speed up and simplify routine tasks. In addition, it offers the potential to explore and create innovative solutions to various problems. Overall, generative AI is a significant technology for the development of various fields, including business and society as a whole.\"})]});export const richText4=/*#__PURE__*/t(i.Fragment,{children:[/*#__PURE__*/e(\"p\",{children:\"As a new school year kicks off, parents, educators, and students alike are inundated with both excitement and apprehension. In the realm of education technology, much of this apprehension stems from the debate surrounding Artificial Intelligence. While the societal conversation often skews toward AI alarmism\u2014warnings about job losses, ethical quandaries, and dystopian futures\u2014it's worth pausing to explore the flip side. I am an AI enthusiast and an educator and I hear too much about which methods can reliably detect if a student generated their essay, and too little about exciting educational opportunities that large language models (LLMs) bring to the classroom. \"}),/*#__PURE__*/e(\"h2\",{children:\"Engaging Students Through Conversations\"}),/*#__PURE__*/e(\"p\",{children:\"Imagine it\u2019s a Physics class. Students could engage in a virtual dialogue with a simulated Sir Isaac Newton, thanks to LLMs. They can ask \u201CSir Isaac\u201D about his contribution to physics and learn about foundational concepts such as inertia, force, and momentum first-hand. The LLM can generate quizzes to test students' understanding, fostering collaboration and critical thinking. For example, \u201CSir Isaac\u201D might offer different questions to different students. The students, in return, could discuss the questions and the right answers with one another and then double check the answers with the teacher. Thus we can fosters independent thinking, effective collaboration and active learning\u2014outcomes that are often hard to achieve in traditional lecture settings.\"}),/*#__PURE__*/e(\"h2\",{children:\"A Hands-On Approach to Algorithms\"}),/*#__PURE__*/e(\"p\",{children:\"Let\u2019s say we want to teach students sorting algorithms as a part of our computer science curriculum. An LLM could generate computer science puzzles that require students to apply sorting algorithms in real-world situations. It's one thing to learn how different algorithms work in theory; it's another to compare and debate solutions testing multiple ones on the flight. This dynamic approach transforms a potentially monotonous class into an engaging, hands-on experience where a student can learn by doing.\"}),/*#__PURE__*/e(\"h2\",{children:\"Broader perspectives\"}),/*#__PURE__*/e(\"p\",{children:\"LLMs could be used to help students structure the creative process. If a student asks the LLM to generate several ideas for their school biology project, they will definitely get some suggestions that hold water. However, these results could be an order of magnitude better. Imagine the students ask the LLM to simulate Jennifer Doudna, a pioneer in CRISPR technology, and brainstorm on the project together. The students can give some ideas that they came up with, ask the model to come up with more options, then give feedback on the ideas that they found exciting. With a few iterations, they might stumble upon something truly genius. Learning can go far beyond acquiring facts; with these tools at their disposal, students have to thinking critically by design.\"}),/*#__PURE__*/e(\"h2\",{children:\"Towards a Future of Responsible AI Integration in Education\"}),/*#__PURE__*/t(\"p\",{children:[\"While there are legitimate concerns about the misuse of AI in educational settings, we shouldn't lose sight of the \",/*#__PURE__*/e(n,{href:\"https://openai.com/blog/teaching-with-ai\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,relValues:[],scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(a.a,{children:\"myriad ways\"})}),\" it can enrich learning experiences. From facilitating deeper engagement in physics to gamifying computer science education, LLMs offer a versatile range of applications. Nevertheless, it's important to temper this enthusiasm with ongoing dialogues around data privacy, accessibility, and ethical considerations.\"]}),/*#__PURE__*/e(\"p\",{children:\"By adopting a balanced approach that recognizes both the potential and the pitfalls of AI, we open the door for a more engaging, personalized, and multidimensional educational experience. In doing so, we move one step closer to realizing a future where AI serves as a valuable collaborator in the educational journey.\"})]});export const richText5=/*#__PURE__*/t(i.Fragment,{children:[/*#__PURE__*/e(\"p\",{children:\"It was only over a decade ago that the virtues of \u201CBig Data\u2019\u2019 echoed in hallways and homes alike. The proliferation of mobile devices, concomitant technologies, social media platforms with user-generated content, and the world becoming more connected than ever before \u2014 have collectively led to a deluge of information (in terms of volume, velocity, and variety). We\u2019re now in a world that is gripped by the extreme narratives surrounding AI boons and banes, but in a reality that lies somewhere in between. A consequence of advances in generative AI and the democratization of large language models is the growing ease with which more information can be generated and consumed. This has critical implications for the propagation of misinformation with potentially damaging ramifications. \"}),/*#__PURE__*/e(\"h2\",{children:\"Claim Check-worthiness\"}),/*#__PURE__*/t(\"p\",{children:[\"To ensure the veracity of information that is created, generated, propagated, and/or consumed by users, automatic fact-checking systems have been developed over the last few years. Claim check-worthiness is an important component of such systems to reduce costs and optimize the use of resources. For instance, it would be computationally expensive to fact-check every single claim in a sea of information on one hand, and it would be rather rudimentary to squander the time of expert fact-checkers to serve this purpose on the other hand. As a result, understanding which claims are worth checking is the first step that is necessary, and this is a task that has gained prominence in recent years. Researchers and practitioners in the natural language processing (NLP) and machine learning communities continue to build systems capable of automatically detecting claims that are \",/*#__PURE__*/e(\"em\",{children:\"checkable\"}),\" and \",/*#__PURE__*/e(\"em\",{children:\"check-worthy\"}),\", and warrant further inspection for their factual accuracy [1\u20133]. A vital ingredient in this process is the availability of labeled data in the context of interest \u2014 be it news articles, social media posts, or discussions on forums. Crowdsourcing has emerged as a scalable means to acquire labeled data by leveraging human input through existing marketplaces such as \",/*#__PURE__*/e(n,{href:\"https://toloka.ai/\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,relValues:[],scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(a.a,{children:\"Toloka AI\"})}),\".\"]}),/*#__PURE__*/e(\"h2\",{children:\"Fact-checking Podcasts \u2014 A New Frontier\"}),/*#__PURE__*/t(\"p\",{children:[\"Ina recent collaboration between Toloka AI and \",/*#__PURE__*/e(n,{href:\"https://www.factiverse.no/\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,relValues:[],scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(a.a,{children:\"Factiverse\"})}),\", a Norwegian company in the fact-checking business, we explored the claim check-worthiness task in the unique context of podcasts. Podcasts are becoming increasingly popular and are estimated to become a 4 billion dollar industry by 2024 [4]. Considering the widespread popularity of podcasting, the growth in platforms, and the increasing amount of content available on a multitude of topics, fact-checking podcast content is extremely important. Limiting the propagation of misinformation via podcasts, safeguarding listeners, and empowering them with the opportunity to make informed decisions by providing factually accurate information is a valuable goal to strive for in this day and age.\"]}),/*#__PURE__*/e(\"h2\",{children:\"Claim check-worthiness of Podcasts Data on Toloka\"}),/*#__PURE__*/t(\"p\",{children:[\"There are a number of intriguing questions surrounding how we can augment information pertaining to the factual accuracy of claims made in podcasts without hampering the listening experience of users. However, the first step remains the effective identification of check-worthy claims. To this end, we considered a random subset of podcasts from the genres of \",/*#__PURE__*/e(\"em\",{children:\"\u201CNews & Politics\u201D\"}),\" or \",/*#__PURE__*/e(\"em\",{children:\"\u201CHealth & Wellness.\u201D\"})]}),/*#__PURE__*/e(\"p\",{children:\"Toloka AI provides a useful set of tools to design and deploy annotation tasks on the platform. It supports the recruiting of crowd workers (referred to as \u2018Tolokers\u2019) from various parts of the world. For the scope of this project, we built a custom annotation interface that Tolokers were directed to use to complete the claim check-worthiness tasks (see Figure 1). \"}),/*#__PURE__*/e(\"img\",{alt:\"figure 1\",className:\"framer-image\",height:\"535\",src:\"https://framerusercontent.com/images/eRK1vwIX4loZEZAHKTMeA1j2FQk.webp\",srcSet:\"https://framerusercontent.com/images/eRK1vwIX4loZEZAHKTMeA1j2FQk.webp?scale-down-to=512 512w,https://framerusercontent.com/images/eRK1vwIX4loZEZAHKTMeA1j2FQk.webp?scale-down-to=1024 1024w,https://framerusercontent.com/images/eRK1vwIX4loZEZAHKTMeA1j2FQk.webp?scale-down-to=2048 2048w,https://framerusercontent.com/images/eRK1vwIX4loZEZAHKTMeA1j2FQk.webp 2560w\",style:{aspectRatio:\"2560 / 1070\"},width:\"1280\"}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"Figure 1: The annotation interface used to assess the check-worthiness of statements in the podcasts considered in this pilot project. Source: Screenshot image by authors.\"})}),/*#__PURE__*/e(\"p\",{children:\"On starting the task and after reading through the instructions, Tolokers are presented with the annotation interface. A panel on the left-hand side of the screen presents the \u201Cstatement\u201D that needs to be assessed for checkworthiness. Tolokers can listen to the audio snippet of the statement from the podcast in addition to reading it if they wish to do so. At the bottom of the panel, Tolokers can scroll through the preceding context (when it exists) to help obtain more context and make an informed assessment when in doubt. The right-hand side of the screen presents the checkworthiness assessment panel.\"}),/*#__PURE__*/e(\"h2\",{children:\"\u201CCheckable\u201D versus \u201CNot Checkable\u201D Statements\"}),/*#__PURE__*/t(\"p\",{children:[\"The first decision that a worker has to make is whether or not the statement is verifiable using publicly available sources (i.e., determine whether the statement is \",/*#__PURE__*/e(\"strong\",{children:\"checkable\"}),\" or \",/*#__PURE__*/e(\"strong\",{children:\"not checkable\"}),\"). Following this decision, the worker needs to provide a rationale for their assessment of check-worthiness. Checkable statements follow one of the four following characterizations:\"]}),/*#__PURE__*/e(\"ol\",{style:{\"--framer-font-size\":\"18px\",\"--framer-text-alignment\":\"start\",\"--framer-text-color\":\"rgb(30, 33, 38)\",\"--framer-text-stroke-width\":\"0px\",\"--framer-text-transform\":\"none\",\"--list-style-type\":\"unset\"},children:/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Factual Descriptions\"}),\": \",/*#__PURE__*/e(\"em\",{children:\"Claims about the existence or characteristics of notable people, places, things, events, or actions, which are possible to verify with public sources\"}),\".\"]})})}),/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Examples\"}),\": \",/*#__PURE__*/e(\"em\",{children:\"\u201CShe won the London Marathon last year.\u201D\"})]}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"\u201CRival groups were involved in a gunfight on the outskirts of the city.\u201D\"})}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"\u201CThe money laundering scheme violated both national and international law.\u201D\"})}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"\u201CItalians drink beer with their pizza, not wine.\u201D\"})}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"\u201CThe budget deficit is the overall amount the government currently owes to its creditors.\u201D\"})}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"\u201CPhotosynthesis is the process in which plants and some other organisms use sunlight to synthesize foods.\u201D\"})}),/*#__PURE__*/e(\"ol\",{start:\"2\",style:{\"--framer-font-size\":\"18px\",\"--framer-text-alignment\":\"start\",\"--framer-text-color\":\"rgb(30, 33, 38)\",\"--framer-text-stroke-width\":\"0px\",\"--framer-text-transform\":\"none\",\"--list-style-type\":\"unset\"},children:/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Cause and Effect\"}),\": *Claims asserting one thing is caused by or linked with another, which can be checked against reputable sources.\"]})})}),/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Examples\"}),\": \",/*#__PURE__*/e(\"em\",{children:\"\u201CThe company collapsed after a rogue employee was discovered to be embezzling funds.\u201D\"})]}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"\u201CSmoking causes cancer.\u201D\"})}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"\u201CObama only got into Harvard because his parents are rich.\u201D\"})}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"\u201CThe new law has led to a rise in crime.\u201D\"})}),/*#__PURE__*/e(\"ol\",{start:\"3\",style:{\"--framer-font-size\":\"18px\",\"--framer-text-alignment\":\"start\",\"--framer-text-color\":\"rgb(30, 33, 38)\",\"--framer-text-stroke-width\":\"0px\",\"--framer-text-transform\":\"none\",\"--list-style-type\":\"unset\"},children:/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Numerical Claims\"}),\": \",/*#__PURE__*/e(\"em\",{children:\"Claims which involve specific statistics or would require counting or analysis of numerical data to verify.\"})]})})}),/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Examples\"}),\": \",/*#__PURE__*/e(\"em\",{children:\"\u201CThe average Mexican consumes more sugar per day than the average American.\u201D\"})]}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"\u201CThe latest poll shows that 80% of people are unhappy with the current government.\u201D\"})}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"\u201CThe drug was found to improve the symptoms of 57% of patients.\u201D\"})}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"\u201CThere are 14 hospitals in the southern region, two more than a decade ago.\u201D\"})}),/*#__PURE__*/e(\"ol\",{start:\"4\",style:{\"--framer-font-size\":\"18px\",\"--framer-text-alignment\":\"start\",\"--framer-text-color\":\"rgb(30, 33, 38)\",\"--framer-text-stroke-width\":\"0px\",\"--framer-text-transform\":\"none\",\"--list-style-type\":\"unset\"},children:/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Quotation\"}),\": \",/*#__PURE__*/e(\"em\",{children:\"Repeating the words of another notable person or entity which can be verified in public sources.\"})]})})}),/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Examples\"}),\": \",/*#__PURE__*/e(\"em\",{children:\"\u201CThe mayor was clear when he said, \u2018All flooded households will receive emergency assistance after a damage assessment.\u2019\u201D\"})]}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"\u201CPresident Roosevelt famously said, \u2018Ich bin ein Berliner.\u2019\u201D\"})}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"\u201CThe company\u2019s CEO announced they are committed to reducing their carbon footprint 50% by 2030.\u201D\"})}),/*#__PURE__*/t(\"p\",{children:[\"Statements that are deemed to be \",/*#__PURE__*/e(\"strong\",{children:\"\u201CNot Checkable\u201D\"}),\" follow one of the following five characterizations:\"]}),/*#__PURE__*/e(\"ol\",{style:{\"--framer-font-size\":\"18px\",\"--framer-text-alignment\":\"start\",\"--framer-text-color\":\"rgb(30, 33, 38)\",\"--framer-text-stroke-width\":\"0px\",\"--framer-text-transform\":\"none\",\"--list-style-type\":\"unset\"},children:/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Not a Claim\"}),\": \",/*#__PURE__*/e(\"em\",{children:\"Not making any sort of claim, including questions not including some factual assertion.\"})]})})}),/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Examples\"}),\": \",/*#__PURE__*/e(\"em\",{children:\"\u201CHello, how are you?\u201D\"})]}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"\u201CHow old are you?\u201D\"})}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"\u201CThanks for chatting with us today.\u201D\"})}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"\u201CI\u2019m sorry, I didn\u2019t know.\u201D\"})}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"\u201CLet\u2019s get into detail.\u201D\"})}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"\u201CStop doing that.\u201D\"})}),/*#__PURE__*/e(\"ol\",{start:\"2\",style:{\"--framer-font-size\":\"18px\",\"--framer-text-alignment\":\"start\",\"--framer-text-color\":\"rgb(30, 33, 38)\",\"--framer-text-stroke-width\":\"0px\",\"--framer-text-transform\":\"none\",\"--list-style-type\":\"unset\"},children:/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Broadcast Details\"}),\": \",/*#__PURE__*/e(\"em\",{children:\"Introducing the speakers, describing the program, or giving details related to the episode contents.\"})]})})}),/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Examples\"}),\": \",/*#__PURE__*/e(\"em\",{children:\"\u201CWelcome to the show, I\u2019m your host, John Smith.\u201D\"})]}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"\u201CToday we\u2019re going to be talking about the history of the internet.\u201D\"})}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"\u201CThis is episode 3 of our series on the history of the internet.\u201D\"})}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"\u201COur guest, Dr. Jane Doe, is joining me in the studio to share her expertise.\u201D\"})}),/*#__PURE__*/e(\"ol\",{start:\"3\",style:{\"--framer-font-size\":\"18px\",\"--framer-text-alignment\":\"start\",\"--framer-text-color\":\"rgb(30, 33, 38)\",\"--framer-text-stroke-width\":\"0px\",\"--framer-text-transform\":\"none\",\"--list-style-type\":\"unset\"},children:/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Emotions and Opinions\"}),\": \",/*#__PURE__*/e(\"em\",{children:\"An emotion that is being felt or expressed, or an opinion that doesn\u2019t contain a checkable factual assertion.\"})]})})}),/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Examples\"}),\": \",/*#__PURE__*/e(\"em\",{children:\"\u201CI love how the tulips look early on a spring morning.\u201D\"})]}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"\u201CHe\u2019s really upset about the way things are going at school.\u201D\"})}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"\u201CThese politicians are the only ones who have half a clue.\u201D\"})}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"\u201CI\u2019m so excited to see you, it\u2019s been too long!\u201D\"})}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"\u201CEveryone around here loves that restaurant.\u201D\"})}),/*#__PURE__*/e(\"ol\",{start:\"4\",style:{\"--framer-font-size\":\"18px\",\"--framer-text-alignment\":\"start\",\"--framer-text-color\":\"rgb(30, 33, 38)\",\"--framer-text-stroke-width\":\"0px\",\"--framer-text-transform\":\"none\",\"--list-style-type\":\"unset\"},children:/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Personal Experience\"}),\": \",/*#__PURE__*/e(\"em\",{children:\"Claims a person makes about their own experience, but which cannot be verified in public sources.\"})]})})}),/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Examples\"}),\": \",/*#__PURE__*/e(\"em\",{children:\"\u201CI passed four empty buses on my way to work yesterday.\u201D\"})]}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"\u201CMy grandmother used lard in her pie crusts.\u201D\"})}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"\u201CI\u2019ve never seen a bluebird in this part of the country.\u201D\"})}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"\u201CMy daughter caught 3 huge trout in that stream last summer.\u201D\"})}),/*#__PURE__*/e(\"ol\",{start:\"5\",style:{\"--framer-font-size\":\"18px\",\"--framer-text-alignment\":\"start\",\"--framer-text-color\":\"rgb(30, 33, 38)\",\"--framer-text-stroke-width\":\"0px\",\"--framer-text-transform\":\"none\",\"--list-style-type\":\"unset\"},children:/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Predictions\"}),\": \",/*#__PURE__*/e(\"em\",{children:\"Claims and predictions about future events or plans that can\u2019t be confirmed at present.\"})]})})}),/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Examples\"}),\": \",/*#__PURE__*/e(\"em\",{children:\"\u201CElon Musk will visit Mars.\u201D\"})]}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"\u201CThe sun will rise tomorrow.\u201D\"})}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"\u201CNew car sales will increase every month going forward.\u201D\"})}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"\u201CThe company will be profitable by the end of the year.\u201D\"})}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"\u201CWe\u2019ll all be dead in 100 years.\u201D\"})}),/*#__PURE__*/t(\"p\",{children:[\"The distinctions between \",/*#__PURE__*/e(\"strong\",{children:\"\u2018Checkable\u2019\"}),\" and \",/*#__PURE__*/e(\"strong\",{children:\"\u2018Not Checkable\u2019\"}),\" statements stem from the practices employed by professional fact-checkers. In this endeavor, we collaborated closely with fact-checkers from Faktisk.no, who regard this classification as the initial step in selecting claims for fact-checking. Faktisk.no AS is a non-profit organization and independent newsroom for fact-checking the social debate and public discourse in Norway. The purpose behind this approach is to effectively sift through sentences and identify those that may not qualify as claims or present difficulties in verification, such as opinions and predictions made by the speakers. By doing so, we ensure that the fact-checking process focuses on statements that can be objectively validated by providing evidence.\"]}),/*#__PURE__*/t(\"p\",{children:[\"When Tolokers determine that a particular statement is \",/*#__PURE__*/e(\"strong\",{children:\"\u2018Checkable\u2019\"}),\", they are asked to (a). highlight the part of the podcast statement that is worth fact-checking on the panel on the left-hand side, and then a new panel appears underneath the assessment panel on the right-hand side, asking the worker to (b). explain the motivation behind why one would want to fact-check the statement by choosing one of the following five options: \"]}),/*#__PURE__*/t(\"ol\",{children:[/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[\"Potential to Cause Harm: \",/*#__PURE__*/e(\"em\",{children:\"I think this statement could cause harm if false.\"})]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[\"Said by a Prominent Person: \",/*#__PURE__*/e(\"em\",{children:\"I want to check if this prominent person actually said this.\"})]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[\"Public Interest: \",/*#__PURE__*/e(\"em\",{children:\"I believe the fact-checking of this claim is for the public interest.\"})]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[\"Surprising: \",/*#__PURE__*/e(\"em\",{children:\"I find this statement surprising, shocking, or otherwise hard to believe.\"})]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[\"Learn More: \",/*#__PURE__*/e(\"em\",{children:\"I would gain new knowledge about this topic by fact-checking this statement.\"})]})})]}),/*#__PURE__*/e(\"img\",{alt:\"figure 2\",className:\"framer-image\",height:\"434\",src:\"https://framerusercontent.com/images/cIA2TuwbfDk4rII9Zd4DunF2k.jpeg\",srcSet:\"https://framerusercontent.com/images/cIA2TuwbfDk4rII9Zd4DunF2k.jpeg?scale-down-to=512 512w,https://framerusercontent.com/images/cIA2TuwbfDk4rII9Zd4DunF2k.jpeg?scale-down-to=1024 1024w,https://framerusercontent.com/images/cIA2TuwbfDk4rII9Zd4DunF2k.jpeg 1586w\",style:{aspectRatio:\"1586 / 868\"},width:\"793\"}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"Figure 2: Highlighting the span of the statement worth fact checking (top-left), specifying the motivation behind the fact checking of the highlighted claim (bottom-right). Source: Screenshot image by authors.\"})}),/*#__PURE__*/e(\"h2\",{children:\"Why is this Useful Anyway?\"}),/*#__PURE__*/e(\"p\",{children:\"In total, we gathered 1,766 annotations from 33 unique Tolokers. We found that the Tolokers annotated approximately 3 statements per minute. The annotations spanned 10 different podcasts, of which 805 statements were found to be checkable and 961 statements were found to be not checkable.\"}),/*#__PURE__*/e(\"p\",{children:\"These annotations play a crucial role in Factiverse\u2019s efforts to enhance and refine their check-worthy claim detection models in their fact-checking pipeline. Additionally, Factiverse serves numerous media customers with podcasting platforms in the Nordics, and these annotations are instrumental in training models to identify check-worthy claims within podcasts. The ultimate goal is to quickly identify podcasts that might contain potentially inaccurate information, along with the claims that are inaccurate and the supporting rationales. This will also solve challenges that fact-checkers such as Faktisk.no face when having to fact-check lengthy podcasts.\"}),/*#__PURE__*/e(\"p\",{children:\"Given that these annotations are fine-grained, including the rationale for considering a claim as check-worthy, they also serve as valuable resources for understanding the model behavior from an explainability perspective. Through the creation of diagnostic datasets, Factiverse can gain insights into the performance of its production models. For instance, they can determine if fact-checking models for numerical claims exhibit any underperformance, enabling Factiverse to take necessary actions to enhance the models\u2019 robustness effectively.\"}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"This article was written with an equal contribution from the CTO of Factiverse, Vinay Setty, and input from other members of the Factiverse and Toloka AI.\"})}),/*#__PURE__*/e(\"h2\",{children:\"References \"}),/*#__PURE__*/t(\"ol\",{children:[/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[\"Guo, Zhijiang, Michael Schlichtkrull, and Andreas Vlachos. \u201CA survey on automated fact-checking.\u201D \",/*#__PURE__*/e(\"em\",{children:\"Transactions of the Association for Computational Linguistics\"}),\" 10 (2022): 178\u2013206.\"]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Wright D, Augenstein I. Claim check-worthiness detection as positive unlabelled learning. arXiv preprint arXiv:2003.02736. 2020 Mar 5.\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Opdahl, Andreas L., Bj\\xf8rnar Tessem, Duc-Tien Dang-Nguyen, Enrico Motta, Vinay Setty, Eivind Throndsen, Are Tverberg, and Christoph Trattner. \u201CTrustworthy journalism through AI.\u201D Data & Knowledge Engineering 146 (2023): 102182.\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(n,{href:\"https://www.theverge.com/2022/5/10/23065056/podcasting-industry-iab-report-audacy-earnings-patreon-pulitzer\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,relValues:[],scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(a.a,{children:\"https://www.theverge.com/2022/5/10/23065056/podcasting-industry-iab-report-audacy-earnings-patreon-pulitzer\"})}),\" \"]})})]})]});export const richText6=/*#__PURE__*/t(i.Fragment,{children:[/*#__PURE__*/t(\"p\",{children:[\"I have always been fascinated by etymology. More often than not, there is an intriguing story behind how words and phrases have acquired the meanings we are so familiar with. Morphing through the ages and mingling with changing times. The \",/*#__PURE__*/e(\"em\",{children:\"Mechanical Turk\"}),\" was a chess-playing humanoid machine made by a Hungarian author and inventor, Wolfgang von Kempelen, in the 18th century. The story goes that Mechanical Turk toured Europe and humbled noteworthy names like Napoleon Bonaparte and Benjamin Franklin in fabled battles of chess. Only later was the secret sauce unveiled in the form of a real human chess whiz hidden away from the naked eye in a cabinet beneath the floorboard, from where they controlled the moves made by the humanoid. \"]}),/*#__PURE__*/t(\"p\",{children:[\"This story was the inspiration behind the naming of the Amazon Mechanical Turk crowdsourcing platform launched in 2005. The platform was designed to solve tasks that could not be solved by contemporary alternatives and required human input or intelligence. It was in this context that the notion of \",/*#__PURE__*/e(\"em\",{children:\"\u201Cartificial artificial intelligence\"}),\"\u201D took shape and form, wherein humans serve as the source of intelligence when it is beyond the capabilities of machines. We have come a long way from there, to the cusp of a brand new notion of \",/*#__PURE__*/e(\"em\",{children:\"\u201Cartificial artificial artificial intelligence\u201D\"}),\". Yes, you read that right. Three artificials. Before you try to crack this walnut with your forehead, let\u2019s take a quick tour down some memory lanes.\"]}),/*#__PURE__*/e(\"h2\",{children:\"The Early Days of Crowdsourcing\"}),/*#__PURE__*/t(\"p\",{children:[\"In his book called \u201CThe Wisdom of Crowds\u2019\u2019 published in 2004, James Surowiecki explored and synthesized the attributes required to form a wise crowd \u2014 one that can often make decisions better than any single individual in the crowd. He identified diversity in opinion, independence in judgment, and decentralized knowledge as vital attributes to that end. In 2006, Jeff Howe coined the term \",/*#__PURE__*/e(\"em\",{children:\"crowdsourcing\"}),\" as a portmanteau of \u2018crowds\u2019 and \u2018outsourcing\u2019 in an article he wrote for Wired magazine on \u201CThe Rise of Crowdsourcing.\u201D He discussed how businesses had begun tapping into the collective capabilities of distributed online communities through open calls to accomplish certain tasks.\"]}),/*#__PURE__*/t(\"p\",{children:[\"Amazon Mechanical Turk thrived after it was first launched, and within years, hundreds of thousands of people around the globe found an opportunity to earn their livelihoods by completing tasks on the platform. This sparked growth in crowdsourcing platforms across the world, forging and cementing a new economy of online microtask crowd work. Researchers and practitioners began to rely on crowdsourcing platforms to accomplish various tasks and demonstrated that even highly complex tasks could be decomposed and crowdsourced. Systems and tools were proposed to support crowd workers in completing tasks effectively. Some prominent examples of contemporary crowdsourcing platforms include \",/*#__PURE__*/e(n,{href:\"https://toloka.ai/\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,relValues:[],scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(a.a,{children:\"Toloka AI\"})}),\" (\u201Ca data-centric environment to support fast and scalable AI development with the help of human insight\u201D) and \",/*#__PURE__*/e(n,{href:\"https://www.prolific.co/\",motionChild:!0,nodeId:\"eo4RAmtig\",openInNewTab:!1,relValues:[],scopeId:\"contentManagement\",smoothScroll:!1,children:/*#__PURE__*/e(a.a,{children:\"Prolific\"})}),\" (a platform to \u201Cconduct research or train the next generation of AI\u201D).\"]}),/*#__PURE__*/e(\"p\",{children:\"In 2009, the release of ImageNet spurred the entire field of machine learning. With over 3.2 million images in 12 subtrees with over five thousand synsets, it was a monumental data collection effort using crowdsourcing via Amazon Mechanical Turk [1]. This provided an unprecedented opportunity for progress in several computer vision tasks, including object recognition and image classification.\"}),/*#__PURE__*/e(\"p\",{children:\"Let us not forget that this progress came with a great set of trials and tribulations. Tremors of the dangers concomitant with relying on human-generated data, prone to cognitive and systematic biases, were felt by many. In 2013 a group of well-known researchers in the crowdsourcing community wrote a paper called \u201CThe Future of Crowd Work,\u201D in which they reflected on the status of the paradigm and the series of challenges that needed immediate addressing [2]. Many of these challenges still remain unsolved a good 10 years later, despite a significant amount of progress. There have been well-documented problems pertaining to the quality of data collected (e.g., propagation of biases), power asymmetry on platforms, abysmal hourly wages, unfair work rejections, invisible labor, unhealthy work environments, and the list goes on. Despite the frailties of what some consider a fractured paradigm of work, remarkable results punctuate the historical timeline, and the power of crowdsourcing has unmistakably contributed to a rate of technological progress only a few would have seen coming.\"}),/*#__PURE__*/e(\"h2\",{children:\"The Intriguing Age of Generative AI\"}),/*#__PURE__*/t(\"p\",{children:[\"Much of the mainstream media around the world today is lost in sweeping narratives around generative AI and what the democratization of large language models can mean. Many more lives will continue to be touched by AI in expected and unexpected ways. And it is the laborious work of humans behind the scenes that has been fuelling this AI revolution in the first place. If we were to \u201Cscrutinize the shadows of AI, we would discover the humans powering it,\u201D as unforgettably put by Mary Gray and Sid Suri in \",/*#__PURE__*/e(\"em\",{children:\"Ghost Work\"}),\" [3].\"]}),/*#__PURE__*/e(\"p\",{children:\"Exaggerated forecasts and clickbait headlines have likened the role of humans in this age to anxious pigeons and equated LLMs to being bold cats \u2014 with the metaphorical cat disrupting the flock, sending them away scattering. But what does the onset of this new age of generative AI models truly mean for human input? Has the need for human input been wiped out for the most part in shaping future technologies? In the remainder of this article, I will argue that the answer to this is a resounding negative and that the main shift we should expect is in the nature of human input that will continue to be needed.\"}),/*#__PURE__*/e(\"p\",{children:\"I recently co-authored a workshop paper exploring how human computation workflows can embrace the emergence of generative AI models [4]. This work was presented at the Generative AI Workshop at the premier HCI conference, ACM CHI 2023, held in Hamburg earlier this year. We highlighted the potential role that large language models (LLMs) can play in augmenting existing crowdsourcing workflows and discussed how such workflows can be empirically evaluated.\"}),/*#__PURE__*/e(\"h2\",{children:\"A Primer on Crowdsourcing Workflows\"}),/*#__PURE__*/t(\"p\",{children:[\"Crowdsourcing workflows are distinct patterns that manage how large-scale tasks are decomposed into smaller tasks to be completed by crowd workers. The crowd-powered word processor, Soylent, applies the \",/*#__PURE__*/e(\"strong\",{children:\"Find-Fix-Verify\"}),\" workflow to produce high-quality text by separating tasks into stages of generating and reviewing text. This enabled \u201Cwriters to call on Mechanical Turk workers to shorten, proofread, and otherwise edit parts of their documents on demand [5].\u201D The \",/*#__PURE__*/e(\"strong\",{children:\"Iterate-and-Vote\"}),\" workflow has been deployed in creating image descriptions, where workers are first asked to write descriptions of images (for example, with an end goal to assist those who are blind). Subsequent voting tasks are then used to converge on an optimal description [6]. The \",/*#__PURE__*/e(\"strong\",{children:\"Map-Reduce\"}),\" workflow has been proposed for \u201Cpartitioning work into tasks that can be done in parallel, mapping tasks to workers, and managing the dependencies between them [7].\u201D Sharing the same essence, tools like \",/*#__PURE__*/e(\"strong\",{children:\"CrowdWeaver\"}),\" have been proposed for managing complex workflows, supporting data sharing between tasks, and providing monitoring tools and real-time task adjustment capability [8].\"]}),/*#__PURE__*/e(\"h2\",{children:\"Boosting Crowdsourcing Workflows with LLMs\"}),/*#__PURE__*/e(\"p\",{children:\"It is unlikely that the emergence of language models renders such workflows, frameworks, and tools completely mundane. On the contrary, the crowdsourcing community is uniquely positioned to embrace the benefits that LLMs can bring by building on decades of research around effective workflows, human-in-the-loop approaches, and knowledge around building hybrid human-AI systems.\"}),/*#__PURE__*/e(\"p\",{children:\"The human-centered perspective of developing technologies focuses on augmenting human experiences in everyday life and amplifying the abilities of people. If LLMs can indeed help crowd workers in completing tasks, they should be embraced and integrated in a fashion that empowers workers to complete tasks more accurately and quickly or in a fashion that improves their overall experience in one way or another.\"}),/*#__PURE__*/t(\"p\",{children:[\"Researchers in information retrieval (a community I have engaged with over the last decade) have recently considered what the proliferation of LLMs can mean for the role of human annotators in the context of relevance judgments for evaluation [9]. They proposed a spectrum of collaboration between humans and LLMs to produce relevance judgments (ranging from human judgments to fully automatic assessments, akin to the popular \",/*#__PURE__*/e(\"em\",{children:\"levels of automation\"}),\"). The authors explored the potential benefits of roping-in LLMs within an assistive capacity for annotation tasks and weighed them in juxtaposition to the risks of doing so. It is clear that LLMs can reduce annotation costs in creating evaluation collections. However, it is unclear whether such collections could be systematically different from those created by humans and how such artifacts would influence the evaluation of information retrieval systems and, thereby, the future design of such systems.\"]}),/*#__PURE__*/t(\"p\",{children:[\"Apart from supporting individual writing or classification tasks within a workflow, researchers are also exploring the application of LLMs in assisting crowd workers. Liu et al. combined the generative power of GPT-3 and the evaluative power of humans to create a new natural language inference dataset that produces more effective models when used as a training set [10]. In a similar vein, others introduced a \",/*#__PURE__*/e(\"em\",{children:\"\u2018Generative Annotation Assistant\u2019\"}),\" to help in the production of dynamic adversarial data collections, significantly improving the rate of collection [11]. However, there are several less-understood open questions pertaining to how LLMs can improve the effectiveness of crowdsourcing workflows and how such workflows can be holistically evaluated.\"]}),/*#__PURE__*/e(\"h2\",{children:\"Many Hurdles Along the Way?\"}),/*#__PURE__*/e(\"p\",{children:\"Much like humans, LLMs can also be prone to bias and unfairness. On one hand, prior work has shown how human annotators fall prey to their own opinions while completing annotation tasks, leading to systematic biases creeping into the resulting data collection [12]. Others have proposed checklists for either combating or reporting potential cognitive biases that may have emerged during the annotation process [13]. On the other hand, recent work has revealed discriminatory stances and stereotypical biases present in LLMs [14, 15].\"}),/*#__PURE__*/e(\"p\",{children:\"The human computation and crowdsourcing research community (HCOMP) has devised a number of effective methods, interfaces, measures, and tools to ensure the collection of high-quality data from crowd workers. It is only a matter of time before we collectively figure out how such quality-related guarantees can be laid out while integrating LLMs in decision-making pipelines.\"}),/*#__PURE__*/e(\"p\",{children:\"On the surface, the integration of LLMs into crowdsourcing workflows can appear to be rather straightforward. As with most proposals for solutions related to complex systems, it is easier said than done. Crowdsourcing has many different stakeholders involved: the task requesters who are keen to gather large-scale annotations, the crowd workers willing to oblige in return for compensation, the platforms providing the infrastructure and serving as the marketplace for these transactions to take place, and indirect end-users of products or technologies that are developed or built in downstream efforts. The impact of including LLMs in workflows has the potential to affect each stakeholder in different ways.\"}),/*#__PURE__*/e(\"p\",{children:\"If crowd workers can become more effective and efficient by leveraging LLMs in intelligent workflows, there is potential to get more work done without increasing costs. However, further work is required to gain a better understanding of the risks and rewards entailing the inclusion of LLMs as a part of crowdsourcing workflows. Who would be responsible for designing, developing, and integrating LLMs into such workflows, considering the potential need for accountability?\"}),/*#__PURE__*/e(\"p\",{children:\"Crowd workers have historically been left to their own devices to improve their productivity and the environments and conditions within which they operate. Shouldn\u2019t it now be the collective responsibility of crowdsourcing platforms and task requesters to better understand how to equip workers with LLM-based solutions that can aid them in successful task completion and improve and augment their work experiences?\"}),/*#__PURE__*/e(\"h2\",{children:\"Artificial Artificial Artificial Intelligence and the Future That Can Be\"}),/*#__PURE__*/t(\"p\",{children:[\"A recent case study explored the extent to which crowdsourced data from \u201Chumans\u201D in a text summarization task was genuinely generated from humans. The authors found evidence to support that over 30% of crowd workers in their study on Amazon Mechanical Turk have already begun to rely on LLMs [16]. Although the study reported these insights from 44 workers alone, and the numbers can be taken with a grain of salt, this does reflect the undeniable prospect of more crowd workers turning towards LLM-based solutions that can help them increase their productivity, maximize their earnings, and improve the time they spend in crowdsourcing marketplaces. This is where the notion of \u201C\",/*#__PURE__*/e(\"strong\",{children:\"artificial\"}),\" artificial artificial intelligence\u201D arises \u2014 crowd workers potentially making use of AI (assistance from LLMs) to provide what is presumably \u201Chuman\u201D input on demand.\"]}),/*#__PURE__*/e(\"img\",{alt:\"Figure: An illustration depicting the emergence of \u201Cartificial artificial artificial intelligence\u201D coined in [29] from AI (1) to AAI (2) and finally AAAI (3). Source: Image by author\",className:\"framer-image\",height:\"1851\",src:\"https://framerusercontent.com/images/YVjOOKs8SWEKq0NptA8zTs1I8us.webp\",srcSet:\"https://framerusercontent.com/images/YVjOOKs8SWEKq0NptA8zTs1I8us.webp?scale-down-to=1024 708w,https://framerusercontent.com/images/YVjOOKs8SWEKq0NptA8zTs1I8us.webp?scale-down-to=2048 1416w,https://framerusercontent.com/images/YVjOOKs8SWEKq0NptA8zTs1I8us.webp 2560w\",style:{aspectRatio:\"2560 / 3702\"},width:\"1280\"}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"Figure: An illustration depicting the emergence of \u201Cartificial artificial artificial intelligence\u201D coined in [29] from AI (1) to AAI (2) and finally AAAI (3). Source: Image by author\"})}),/*#__PURE__*/e(\"p\",{children:\"Further consideration is needed regarding the transparency and explainability of LLMs compared to what can be elicited from humans. When crowd workers complete tasks such as annotation or others that require decision-making, task requesters can extract meaningful rationales through follow-up questions. Crowd workers have the wherewithal to provide such insights where needed. The same cannot be currently achieved with LLMs. Yes, there are methods for model explainability, but none have demonstrated a level of effectiveness on par with what can be achieved with humans at both ends of the line. This perception of LLMs as a \u201Cblack box\u201D can create barriers to adoption for task requesters and crowdsourcing platforms, while also impeding the appropriate reliance of crowd workers on such tools.\"}),/*#__PURE__*/e(\"p\",{children:\"Humans and LLMs? There is an endless stream of possibilities with a sea of intriguing questions and only a handful of glimmering answers. Seizing the opportunity to integrate this technological advancement to improve crowd work is less like stirring a hornet\u2019s nest and more like catching a gust of wind in our sails. Let us get busy, for a beautiful future awaits when we can shape it with humans taking center stage.\"}),/*#__PURE__*/e(\"h2\",{children:\"References\"}),/*#__PURE__*/t(\"ol\",{style:{\"--framer-font-size\":\"18px\",\"--framer-text-alignment\":\"start\",\"--framer-text-color\":\"rgb(30, 33, 38)\",\"--framer-text-stroke-width\":\"0px\",\"--framer-text-transform\":\"none\",\"--list-style-type\":\"unset\"},children:[/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[\"Deng, J., Dong, W., Socher, R., Li, L. J., Li, K., & Fei-Fei, L. (2009, June). Imagenet: A large-scale hierarchical image database. In \",/*#__PURE__*/e(\"em\",{children:\"2009 IEEE Conference on Computer Vision and Pattern Recognition\"}),\" (pp. 248\u2013255). IEEE.\"]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[\"Kittur, A., Nickerson, J.V., Bernstein, M., Gerber, E., Shaw, A., Zimmerman, J., Lease, M. and Horton, J., 2013, February. The future of crowd work. In \",/*#__PURE__*/e(\"em\",{children:\"Proceedings of the 2013 conference on Computer supported cooperative work\"}),\" (pp. 1301\u20131318).\"]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[\"Gray, M. L., & Suri, S. (2019). \",/*#__PURE__*/e(\"em\",{children:\"Ghost work: How to stop Silicon Valley from building a new global underclass\"}),\". Eamon Dolan Books.\"]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[\"Allen, G., He, G., Gadiraju, U. Power-up! What Can Generative Models Do for Human Computation Workflows? In \",/*#__PURE__*/e(\"em\",{children:\"Proceedings of the Generative AI Workshop at ACM International Conference on Human Factors in Computing Systems\"}),\" (CHI 2023).\"]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[\"Bernstein, Michael S., Greg Little, Robert C. Miller, Bj\\xf6rn Hartmann, Mark S. Ackerman, David R. Karger, David Crowell, and Katrina Panovich. \u201CSoylent: a word processor with a crowd inside.\u201D In \",/*#__PURE__*/e(\"em\",{children:\"Proceedings of the 23rd annual ACM symposium on User Interface Software and Technology\"}),\", pp. 313\u2013322. 2010.\"]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[\"Little, G., Chilton, L. B., Goldman, M., & Miller, R. C. (2009, June). Turkit: Tools for Iterative Tasks on Mechanical Turk. In \",/*#__PURE__*/e(\"em\",{children:\"Proceedings of the ACM SIGKDD workshop on human computation\"}),\" (pp. 29\u201330).\"]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[\"Kittur, A., Smus, B., Khamkar, S., & Kraut, R. E. (2011, October). Crowdforge: Crowdsourcing complex work. In \",/*#__PURE__*/e(\"em\",{children:\"Proceedings of the 24th annual ACM symposium on User interface software and technology\"}),\" (pp. 43\u201352).\"]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[\"Kittur, A., Khamkar, S., Andr\\xe9, P. and Kraut, R., 2012, February. CrowdWeaver: visually managing complex crowd work. In \",/*#__PURE__*/e(\"em\",{children:\"Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work\"}),\" (pp. 1033\u20131036).\"]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[\"Faggioli, G., Dietz, L., Clarke, C., Demartini, G., Hagen, M., Hauff, C., Kando, N., Kanoulas, E., Potthast, M., Stein, B. and Wachsmuth, H., 2023. Perspectives on Large Language Models for Relevance Judgment. \",/*#__PURE__*/e(\"em\",{children:\"arXiv preprint arXiv:2304.09161\"}),\".\"]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[\"Liu, Z., Roberts, R.A., Lal-Nag, M., Chen, X., Huang, R. and Tong, W., 2021. AI-based language models powering drug discovery and development. \",/*#__PURE__*/e(\"em\",{children:\"Drug Discovery Today\"}),\", 26(11), pp.2593\u20132607.\"]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[\"Bartolo, M., Thrush, T., Riedel, S., Stenetorp, P., Jia, R. and Kiela, D., 2021. Models in the loop: Aiding crowd workers with generative annotation assistants. \",/*#__PURE__*/e(\"em\",{children:\"arXiv preprint arXiv:2112.09062\"}),\".\"]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[\"Hube, C., Fetahu, B. and Gadiraju, U., 2019, May. Understanding and mitigating worker biases in the crowdsourced collection of subjective judgments. In \",/*#__PURE__*/e(\"em\",{children:\"Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems\"}),\" (pp. 1\u201312).\"]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[\"Draws, T., Rieger, A., Inel, O., Gadiraju, U., & Tintarev, N. (2021, October). A checklist to combat cognitive biases in crowdsourcing. In \",/*#__PURE__*/e(\"em\",{children:\"Proceedings of the AAAI conference on human computation and crowdsourcing\"}),\" (Vol. 9, pp. 48\u201359).\"]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[\"Abid, A., Farooqi, M. and Zou, J., 2021, July. Persistent anti-muslim bias in large language models. In \",/*#__PURE__*/e(\"em\",{children:\"Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society\"}),\" (pp. 298\u2013306).\"]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[\"Nadeem, M., Bethke, A. and Reddy, S., 2020. StereoSet: Measuring stereotypical bias in pre-trained language models. \",/*#__PURE__*/e(\"em\",{children:\"arXiv preprint arXiv:2004.09456\"}),\".\"]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[\"Veselovsky, V., Ribeiro, M. H., & West, R. (2023). Artificial Artificial Artificial Intelligence: Crowd Workers Widely Use Large Language Models for Text Production Tasks. \",/*#__PURE__*/e(\"em\",{children:\"arXiv preprint arXiv:2306.07899\"}),\".\"]})})]})]});export const richText7=/*#__PURE__*/t(i.Fragment,{children:[/*#__PURE__*/e(\"h2\",{children:\"About the client \"}),/*#__PURE__*/e(\"p\",{children:\"Our client is a content and news sharing platform with more than 70M monthly active users. One of the popular features is public group chats, where users can share text messages, images, and videos. All user-generated content needs to pass moderation to make sure it complies with the platform\u2019s guidelines. \"}),/*#__PURE__*/e(\"h2\",{children:\"Challenge\"}),/*#__PURE__*/e(\"p\",{children:\"The client was already using an internal workforce to handle content moderation. As the platform grew in popularity, this workforce couldn\u2019t keep up \u2014 they needed a scalable solution to moderate non-compliant public channels by monitoring messages and threads\u200B. The goal was to ensure a safe environment and protect users from fraud.\"}),/*#__PURE__*/e(\"img\",{alt:\"Challenge\",className:\"framer-image\",height:\"903\",src:\"https://framerusercontent.com/images/XQwq20OJ18jyP6ndiGWbnZ0frQ.webp\",srcSet:\"https://framerusercontent.com/images/XQwq20OJ18jyP6ndiGWbnZ0frQ.webp?scale-down-to=512 512w,https://framerusercontent.com/images/XQwq20OJ18jyP6ndiGWbnZ0frQ.webp?scale-down-to=1024 1024w,https://framerusercontent.com/images/XQwq20OJ18jyP6ndiGWbnZ0frQ.webp?scale-down-to=2048 2048w,https://framerusercontent.com/images/XQwq20OJ18jyP6ndiGWbnZ0frQ.webp 2560w\",style:{aspectRatio:\"2560 / 1807\"},width:\"1280\"}),/*#__PURE__*/e(\"h2\",{children:\"Solution\"}),/*#__PURE__*/e(\"p\",{children:\"Toloka developed adaptive ML models for automated moderation in 7 classes and 3 languages \u2014 English, Russian, and Ukrainian.\"}),/*#__PURE__*/e(\"p\",{children:\"The solution detects these classes in text, images, and videos:\"}),/*#__PURE__*/t(\"ul\",{style:{\"--framer-font-size\":\"18px\",\"--framer-text-alignment\":\"start\",\"--framer-text-color\":\"rgb(30, 33, 38)\",\"--framer-text-stroke-width\":\"0px\",\"--framer-text-transform\":\"none\"},children:[/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Offensive speech\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Clickbait\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Money transfer\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Spam\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Selling activity\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Personal data\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Illegal content\"})})]}),/*#__PURE__*/e(\"p\",{children:\"We use a human-in-the-loop pipeline to ensure quality. Toloka\u2019s crowd of qualified annotators evaluate the accuracy and precision of model output and provide feedback for additional model training to continually improve the model output. \"}),/*#__PURE__*/e(\"img\",{alt:\"Solution\",className:\"framer-image\",height:\"1162\",src:\"https://framerusercontent.com/images/lktDe4u2loumMNrQI9Zopqv0Jnk.webp\",srcSet:\"https://framerusercontent.com/images/lktDe4u2loumMNrQI9Zopqv0Jnk.webp?scale-down-to=512 512w,https://framerusercontent.com/images/lktDe4u2loumMNrQI9Zopqv0Jnk.webp?scale-down-to=1024 1024w,https://framerusercontent.com/images/lktDe4u2loumMNrQI9Zopqv0Jnk.webp?scale-down-to=2048 2048w,https://framerusercontent.com/images/lktDe4u2loumMNrQI9Zopqv0Jnk.webp 2560w\",style:{aspectRatio:\"2560 / 2324\"},width:\"1280\"}),/*#__PURE__*/e(\"p\",{children:\"If a user reports inappropriate content, it is sent directly to Toloka for manual moderation. Annotators label the content with one of the categories above, or mark it as compliant. Depending on the verdict, the content is removed from the platform or approved. \"}),/*#__PURE__*/e(\"h2\",{children:\"Business impact\"}),/*#__PURE__*/e(\"p\",{children:\"Toloka\u2019s solution detects 30% more non-compliant content than the previous in-house solution, processing 10-20K messages per day. The automated solution is much faster, taking about 5 minutes to verify one item. Moderation costs dropped by 40% for the client. At the same time, user retention increased by 20% due to better user experience with less fraud.\"}),/*#__PURE__*/e(\"p\",{children:\"Toloka Moderation offers out-of-the-box solutions and enterprise customization for efficient content moderation to protect your users. \"})]});export const richText8=/*#__PURE__*/t(i.Fragment,{children:[/*#__PURE__*/e(\"p\",{children:\"For those who are new to the field of artificial intelligence, grasping the many complex terms associated with it can prove to be quite overwhelming. Artificial Intelligence (AI), Machine Learning (ML), Large Language Models (LLMs), and Generative AI are all related concepts in the field of computer science, but there are important distinctions between them. They have significant differences in their functionality and applications. We will take a closer look at these concepts and gain a better understanding of their distinctions further.\"}),/*#__PURE__*/e(\"h2\",{children:\"Artificial Intelligence\"}),/*#__PURE__*/e(\"p\",{children:\"AI belongs to the field of computer science that deals with the development of computer systems that can perform tasks that typically require human intelligence, such as speech recognition, natural language processing (NLP), text generation and translation, video, sound, and image generation, decision making, planning, and more.\"}),/*#__PURE__*/e(\"p\",{children:\"AI, in general, refers to the development of intelligent systems that can mimic human behavior and decision-making processes. It encompasses techniques and approaches enabling machines to perform tasks, analyze visual and textual data, and respond or adapt to their environment. One of the key advantages of artificial intelligence is its ability to process large amounts of data and find patterns in it. AI tools are designed to make decisions or take actions based on that knowledge.\"}),/*#__PURE__*/e(\"p\",{children:\"AI has applications in many fields including marketing, medicine, finance, science, education, industry, and many others. For example, in marketing it is applied to generate marketing materials, in medicine it is utilized to diagnose diseases, and in finance, it is used to analyze financial markets and make investment decisions.\"}),/*#__PURE__*/e(\"p\",{children:\"There are a handful of types and classifications of AI, including one based on the so-called AI evolution. According to this hypothetical evolution classification, all forms of AI existing now are considered weak AI because they are limited to a specific or narrow area of cognition. Weak AI lacks human consciousness, although it can simulate it in some situations.\"}),/*#__PURE__*/e(\"p\",{children:\"The next stage of AI development may be a conceptual (so far) form called strong AI or artificial general intelligence, endowed with human consciousness and capable of performing human tasks, constructing mental abilities, reasoning, and learning from experience. It will no longer \u201Cmimic\u201D human behavior, it will practically become a real thinking being.\"}),/*#__PURE__*/e(\"p\",{children:\"The peak of AI development may result in Super AI, which would outperform humans in all areas and may even become the cause of human extinction. But for now, this is only a hypothesis. Artificial Intelligence can also be categorized into discriminative and generative.\"}),/*#__PURE__*/e(\"h2\",{children:\"Discriminative and Generative AI\"}),/*#__PURE__*/e(\"p\",{children:\"Discriminative and generative AI are two different approaches to building AI systems. Discriminative AI focuses on learning the boundaries that separate different classes or categories in the training data. These models do not aim to generate new samples, but rather to classify or label input data based on what class it belongs to. Discriminative models are trained to identify the patterns and features that are specific to each class and make predictions based on those patterns.\"}),/*#__PURE__*/e(\"p\",{children:\"Discriminative models are often used for tasks like classification or regression, sentiment analysis, and object detection. Examples of discriminative AI include algorithms like logistic regression, decision trees, random forests and so on.\"}),/*#__PURE__*/e(\"p\",{children:\"In contrast to discriminative AI, Generative AI focuses on building models that can generate new data similar to the training data it has seen. Generative models learn the underlying probability distribution of the training data and can then generate new samples from this learned distribution.\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI tools are capable of image synthesis, text generation, or even music. Such systems typically involve deep learning and neural networks to learn patterns and relationships in the training data. They use that knowledge to create new content. Examples of generative AI models include Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), transformer and diffusion models, and many more.\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI is inconceivable without foundation models, that play a significant role in advancing it. They are large-scale algorithms that serve as the backbone of AI systems. By leveraging the learned knowledge of foundation models, generative AI systems can generate high-quality and contextually relevant content. These models have seen tremendous progress recently, allowing them to generate human-like text, answer questions, write essays, create stories, and much more.\"}),/*#__PURE__*/e(\"p\",{children:\"Through the utilization of a foundational model, we have the capacity to craft more specialized and advanced models that are specifically designed for particular domains or use cases. For instance, generative AI can utilize foundation models as a core for creating large language models. By leveraging the knowledge learned from training on vast amounts of text data, generative AI can generate coherent and contextually relevant text, often resembling human-generated content.\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI, which can generate new content or create new information, is becoming increasingly valuable in today's business landscape. It can be used to create high-quality marketing materials, and various business documents ranging from official email templates to annual reports, social media posts, product descriptions, articles, and so on. Generative AI can help businesses automate content creation and achieve scalability without compromising on quality. Such systems are already being incorporated into numerous business applications.\"}),/*#__PURE__*/e(\"h2\",{children:\"Machine Learning vs. AI\"}),/*#__PURE__*/e(\"p\",{children:\"Machine Learning is a specific subset or application of AI that focuses on providing systems the ability to learn and improve from experience without being explicitly programmed. ML is a critical component of many AI systems. ML algorithms are used to train AI models by providing them with datasets containing labeled examples or historical data. The model then learns the underlying patterns in the training data, enabling it to make accurate predictions or decisions on new, unseen data. By continuously feeding data to ML models, they can adapt and improve their performance over time.\"}),/*#__PURE__*/e(\"p\",{children:\"AI encompasses the broader concept of developing intelligent machines, while ML focuses on training systems to learn and make predictions from data. AI aims to replicate human-like behavior, while ML enables machines to automatically learn patterns from data.\"}),/*#__PURE__*/e(\"p\",{children:\"A machine learning model in AI is a mathematical representation or algorithm that is trained on a dataset to make predictions or take actions without being explicitly programmed. It is a fundamental component of AI systems as it enables computers to learn from data and improve performance over time.\"}),/*#__PURE__*/e(\"h2\",{children:\"Generative AI vs. Large Language Models\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI is a broad concept encompassing various forms of content generation, while LLM is a specific application of generative AI. Large language models serve as foundation models, providing a basis for a wide range of natural language processing (NLP) tasks. Generative AI can encompass a range of tasks beyond language generation, including image and video generation, music composition, and more. Large language models, as one specific application of generative AI, are specifically designed for tasks revolving around natural language generation and comprehension.\"}),/*#__PURE__*/e(\"p\",{children:\"Large language models operate by using extensive datasets to learn patterns and relationships between words and phrases. They have been trained on vast amounts of text data to learn the statistical patterns, grammar, and semantics of human language. This vast amount of text may be taken from the Internet, books, and other sources to develop a deep understanding of human language.\"}),/*#__PURE__*/e(\"p\",{children:\"An LLM can take a given input (a sentence or a prompt) and generate a response: coherent and contextually relevant sentences or even paragraphs based on a given prompt or input. The model uses various techniques, including attention mechanisms, transformers, and neural networks, to process the input and generate an output that aims to be coherent and contextually appropriate.\"}),/*#__PURE__*/e(\"p\",{children:\"Both generative AI and large language models involve the use of deep learning and neural networks. While generative AI aims to create original content across various domains, large language models specifically concentrate on language-based tasks and excel in understanding and generating human-like text.\"}),/*#__PURE__*/e(\"h2\",{children:\"Large Language Models applications\"}),/*#__PURE__*/e(\"p\",{children:\"Large language models can perform a wide range of language tasks, including answering questions, writing articles, translating languages, and creating conversational agents, making them extremely valuable tools for various industries and applications.\"}),/*#__PURE__*/e(\"p\",{children:\"By providing prompt or specific instructions, developers can utilize these large language models as code generation tools to write code snippets, functions, or even entire programs. This can be useful for automating repetitive tasks, prototyping, or exploring new ideas quickly.\"}),/*#__PURE__*/e(\"p\",{children:\"Code generation with large language models has the potential to greatly assist developers, saving time and effort in generating boilerplate code, exploring new techniques, or assisting with knowledge transfer. However, it's important to judiciously use these models in software development, validate the output, and maintain a balance between automation and human expertise.\"}),/*#__PURE__*/e(\"p\",{children:\"Companies are employing large language models to develop intelligent chatbots. They can enhance customer service by offering quick and accurate responses, improving customer satisfaction, and reducing human workload.\"}),/*#__PURE__*/e(\"p\",{children:\"Large language models can help businesses automate content creation processes, as well as save time and resources. Additionally, language models assist in content arrangement by analyzing and summarizing large volumes of information from various sources.\"}),/*#__PURE__*/e(\"p\",{children:\"Businesses process and analyze unstructured text data more effectively with the help of large language models. They can fulfill tasks like text classification, information extraction, sentiment analysis, and more. All of this plays a big role in understanding customer behavior and predicting market trends.\"}),/*#__PURE__*/e(\"h2\",{children:\"Popular LLMs\"}),/*#__PURE__*/e(\"p\",{children:\"Here are some popular large language models which have revolutionized many NLP tasks and have applications in chatbots, virtual assistants, content creation, and machine translation, among others:\"}),/*#__PURE__*/e(\"h3\",{children:\"GPT-4\"}),/*#__PURE__*/e(\"p\",{children:\"Developed by OpenAI, GPT-4 is one of the largest publicly available LLM models. It is a language model which is an extension of the GPT-3. It has been trained on a large amount of data and has higher accuracy and ability to generate text than previous models. The system can read, analyze or generate up to 25,000 words of text. The exact number of GPT-4 parameters is unknown, but according to some researchers it has approximately 1.76 trillion of them.\"}),/*#__PURE__*/e(\"h3\",{children:\"GLaM (Generalist Language Model)\"}),/*#__PURE__*/e(\"p\",{children:\"GLaM is an advanced conversational AI model with 1.2 trillion parameters developed by Google. It is designed to generate human-like responses to user prompts and simulate text-based conversations. GLaM is trained on a wide range of internet text data, making it capable of understanding and generating responses on various topics. It aims to produce coherent and contextually relevant responses, leveraging the vast knowledge it has learned from its training data.\"}),/*#__PURE__*/e(\"h3\",{children:\"BERT (Bidirectional Encoder Representations from Transformers)\"}),/*#__PURE__*/e(\"p\",{children:\"Developed by Google, BERT is another widely-used LLM model with 340 million parameters. BERT is a pre-trained model that excels at understanding and processing natural language data. It has been used in various applications, including text classification, entity recognition, and question-answering systems.\"}),/*#__PURE__*/e(\"h3\",{children:\"LLaMA (Large Language Model Meta AI)\"}),/*#__PURE__*/e(\"p\",{children:\"LLaMA (Large Language Model Meta AI) NLP model with billions of parameters and trained in 20 languages released by Meta. The model is accessible to all for non-commercial use. LLaMA has the capability to have conversations and engage in creative writing, making it a versatile language model.\"}),/*#__PURE__*/e(\"p\",{children:\"Overall, the operation of LLMs involves complex computations and sophisticated algorithms to generate coherent and contextually relevant text based on the given input. Such systems have a wide range of applications, including text completion, translation, chatbots, content generation, and more.\"}),/*#__PURE__*/e(\"h2\",{children:\"Conclusion\"}),/*#__PURE__*/e(\"p\",{children:\"Artificial Intelligence (AI), Machine Learning (ML), Large Language Models (LLMs), and Generative AI are all related concepts in the field of computer science, but there are important distinctions between them. Understanding the differences between these terms is crucial as they represent different vital aspects and features in AI.\"}),/*#__PURE__*/e(\"p\",{children:\"In summary, AI is a broad field covering the development of systems that simulate intelligent behavior. It encompasses various techniques and approaches, while machine learning is a subfield of AI that focuses on designing algorithms that enable systems to learn from data. Large language models are a specific type of ML model trained on text data to generate human-like text, and generative AI refers to the broader concept of AI systems capable of generating various types of content.\"}),/*#__PURE__*/e(\"p\",{children:\"ML, LLMs, Generative AI: these are just a few of the many terms used in AI. Gaining insight into these distinctions is essential for comprehending the unique characteristics and uses of AI, ML, LLMs, and Generative AI within the constantly changing world of technology. As the AI landscape continues to evolve, new concepts will inevitably appear and the terminology we employ to characterize these systems will transform in the future.\"}),/*#__PURE__*/e(\"h2\",{children:\"About Toloka\"}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"em\",{children:\"Toloka is a European company based in Amsterdam, the Netherlands that provides data for Generative AI development. Toloka empowers businesses to build high quality, safe, and responsible AI. We are the trusted data partner for all stages of AI development from training to evaluation. Toloka has over a decade of experience supporting clients with its unique methodology and optimal combination of machine learning technology and human expertise, offering the highest quality and scalability in the market.\"})})]});export const richText9=/*#__PURE__*/t(i.Fragment,{children:[/*#__PURE__*/e(\"p\",{children:\"Generative artificial intelligence is gradually becoming an integral part of our everyday life. It is of crucial importance to businesses, and many companies have already embraced generative AI initiatives. It already helps to accomplish many business tasks and maximize employee productivity as well as work efficiency.\"}),/*#__PURE__*/e(\"p\",{children:\"For instance, generative AI is capable of creating draft documents, preparing replies to incoming emails, generating marketing materials, and much more. Machines are handling routine tasks faster and with greater efficiency, freeing up human workforce resources for far more interesting and meaningful tasks.\"}),/*#__PURE__*/e(\"p\",{children:\"Such a rapid increase in the use of generative AI is largely driven by the advancement of modern computers and deep learning. In the future, generative AI will be able to accomplish even more tasks and become even more accessible. Further, we will explore how Generative AI works, its purpose, and why it is so relevant.\"}),/*#__PURE__*/e(\"h2\",{children:\"Generative AI explained\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI represents a variety of artificial intelligence techniques that can create new content. The data that such AI generates may be completely different, ranging from source code to music.\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI has generative models at its core and these artificial intelligence algorithms learn to generate new content, such as images or text, derived from regularities they have discovered in existing data. For instance, models that predict the next word in a sequence are generally generative since they can produce entire sentences.\"}),/*#__PURE__*/e(\"p\",{children:\"For the purpose of creating unique content, effective generative AI models employ deep neural networks. Deep learning generative AI has seen huge development recently as computers have become more accessible and can now process vast amounts of complex data.\"}),/*#__PURE__*/e(\"p\",{children:\"Today, immense amounts of data can be processed with the help of so-called foundation models. Such models are the backbone of complicated AI systems and can perform several tasks simultaneously. Foundation models in generative AI consist of neural networks with multiple layers of neurons, which are created in the likeness of the human brain. Due to such architecture, generative AI is considered a deep learning method that can process gigantic sets of data and carry out more than one task at a time.\"}),/*#__PURE__*/e(\"p\",{children:\"Such models have already significantly impacted fields such as computer vision and human language processing, enabling anything from creating realistic images to language translation.\"}),/*#__PURE__*/e(\"h2\",{children:\"Discriminative and Generative AI\"}),/*#__PURE__*/e(\"p\",{children:\"Two classes of algorithms are employed in machine learning: discriminative and generative models. As previously discussed, generative AI heavily relies on generative AI models. The goal of a generative model is to produce new patterns from what has already been allocated in the training data. This requires the generative model to grasp the structural basis of the data and analyze a realistic generalized representation of the dataset. This kind of solution produces unique text, realistic images, music, source code, and even 3D models.\"}),/*#__PURE__*/e(\"p\",{children:\"The goal of discriminative models, so-called traditional AI, is to master recognizing the desired values from a labeled training set. Simply put, they learn to detect patterns in data sets and apply them to fulfill a task of new patterns prediction or classification.\"}),/*#__PURE__*/e(\"h2\",{children:\"How does generative AI work?\"}),/*#__PURE__*/e(\"p\",{children:\"Sophisticated AI algorithms, which incorporate neural networks, are utilized by generative AI tools to create synthetic data from existing content. Each type of generative AI model works differently and has a unique process of content creation.\"}),/*#__PURE__*/e(\"h3\",{children:\"Types of generative AI models\"}),/*#__PURE__*/e(\"p\",{children:\"Here are the most notable types of generative AI models:\"}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"strong\",{children:\"Generative Adversarial Networks (GANs)\"})}),/*#__PURE__*/e(\"p\",{children:\"The generative adversarial network is composed of two subnets, one that generates values and one that discriminates. They are playing a game called a zero-sum game, which consists in beating each other. Zero-sum signifies that whenever the discriminator successfully identifies real or counterfeit samples, it is rewarded with no modifications for its parameters, meanwhile, the generator gets large updates to its model parameters as a punishment.\"}),/*#__PURE__*/e(\"p\",{children:\"The generator produces images, and the discriminator tries to guess if it's a real picture. If the discriminator indicates that it is genuine, then the generator wins, as the picture is not real anyway, although it looks like real, human-generated content.\"}),/*#__PURE__*/e(\"p\",{children:\"Thus the mechanism of mutual evolution is implemented. The failures and victories of these two neural networks help them in their training. As a result of successful learning, the generative part of the network may produce a variety of values that fulfill the requirements for the resulting value, for example, producing different images similar to those in the training set.\"}),/*#__PURE__*/e(\"p\",{children:\"Here is a summary of the GAN's stages of operation. Two neural networks, a generator, and a discriminator, are trained together:\"}),/*#__PURE__*/t(\"ol\",{style:{\"--framer-font-size\":\"18px\",\"--framer-text-alignment\":\"start\",\"--framer-text-color\":\"rgb(30, 33, 38)\",\"--framer-text-stroke-width\":\"0px\",\"--framer-text-transform\":\"none\",\"--list-style-type\":\"unset\"},children:[/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"The generator creates a series of samples and these, along with genuine examples from the source dataset, are provided to the discriminator;\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"The discriminator classifies them as genuine or fake;\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"The discriminator is updated if it has failed and misclassified the generated content to improve the recognition of genuine or forged samples in the next training round;\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"The generator is updated based on whether the generated samples tricked the discriminator or not;\"})})]}),/*#__PURE__*/e(\"p\",{children:\"Apart from the conventional so-called Vanilla GAN, the network has evolved and acquired improvements over the years. There are several types of GAN:\"}),/*#__PURE__*/t(\"ul\",{style:{\"--framer-font-size\":\"18px\",\"--framer-text-alignment\":\"start\",\"--framer-text-color\":\"rgb(30, 33, 38)\",\"--framer-text-stroke-width\":\"0px\",\"--framer-text-transform\":\"none\"},children:[/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"DCGAN (Deep Convolutional Generative Adversarial Network) is more stable for training and can generate higher-quality outputs\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Improved DCGAN allows the generation of higher-resolution images\"})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/e(\"p\",{children:\"Conditional GAN uses additional information, such as higher quality outcomes, as well as the ability to have some degree of control over how the generated images will look like\"})})]}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"strong\",{children:\"Transformer-based models\"})}),/*#__PURE__*/e(\"p\",{children:\"Transformer is a model that employs an attention mechanism to increase the speed of training. It is a relatively new type of neural network that can translate text, write papers, and create software code. It is currently one of the most advanced systems in the field of natural language processing (NLP).\"}),/*#__PURE__*/e(\"p\",{children:\"Transformer neural networks can be parallelized and trained much faster, as they do not process sequential data in order, like recurrent neural networks, for example. In that way, such network architecture escapes recurrence, i.e., sequential computation. The calculations happen simultaneously, making the network operate faster.\"}),/*#__PURE__*/e(\"p\",{children:\"The key constituents of Transformers are the encoder and decoder. The encoder and decoder consist of layers. The encoder converts the original input data and transforms it into a vector. The decoder, on its part, interprets it in the form of a new sequence. The basic idea behind Transformer is to use self-adaptive attention to process a sequence of input data. This allows the model to take into account the context of each word and associate it with other words in the sequence. Due to the attention mechanism, the data in Transformer goes through a shorter route in comparison to the recurrent approach. Instead, it focuses on remote yet significant words and processes these words straight away. Consequently, the neural network has improved long-term memory. The renowned Generative Pre-trained Transformer (GPT) text generator, first introduced by the research organization OpenAI in 2018, belongs to this type of generative AI model.\"}),/*#__PURE__*/e(\"p\",{children:/*#__PURE__*/e(\"strong\",{children:\"Diffusion models\"})}),/*#__PURE__*/e(\"p\",{children:\"Diffusion models are a subcategory of deep generative models, which are comprised of stages of forward and backward diffusion. They generate unique data similar to the data they are trained on. They slowly add accidental noise to the data, thereafter learning ways to reverse the diffusion process to create the desired data samples from the noise.\"}),/*#__PURE__*/e(\"p\",{children:\"The diffusion models operation includes two major steps:\"}),/*#__PURE__*/t(\"ol\",{style:{\"--framer-font-size\":\"18px\",\"--framer-text-alignment\":\"start\",\"--framer-text-color\":\"rgb(30, 33, 38)\",\"--framer-text-stroke-width\":\"0px\",\"--framer-text-transform\":\"none\",\"--list-style-type\":\"unset\"},children:[/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Forward diffusion\"}),\", which consists of distorting the training data by adding Gaussian noise step by step and eliminating the details until the data is converted to plain noise;\"]})}),/*#__PURE__*/e(\"li\",{\"data-preset-tag\":\"p\",children:/*#__PURE__*/t(\"p\",{children:[/*#__PURE__*/e(\"strong\",{children:\"Reverse diffusion\"}),\", which consists of training the neural network to reverse the diffusion process, which means restoring the original image by synthesizing pure noise by gradually reducing the noise until a new clean result is obtained.\"]})})]}),/*#__PURE__*/e(\"p\",{children:\"In other words, diffusion models are trained on a sample of hundreds of thousands of images. Then at each step a bit of noise of a certain value is imposed on the image from the sample, whereas the model learns to reverse this noise, hence making the quality of the image higher.\"}),/*#__PURE__*/e(\"p\",{children:\"Data scientists then proceed to apply the model trained in this way to an entirely new image entirely made up of totally irregular noise, where at each step by reversing the noise, the model will be capable of forming an entirely novel picture, gradually getting rid of the random noise via reverse diffusion.Generative AI tools such as DALL-E, MidJourney, and Stable Diffusion use such models to produce realistic images.\"}),/*#__PURE__*/e(\"h3\",{children:\"Variational Autoencoders (VAEs)\"}),/*#__PURE__*/e(\"p\",{children:\"Variational autoencoders are comprised of two networks called an encoder and a decoder. The encoder transforms the input data to a smaller and denser representation of the data, called the latent space. This condensed representation stores the information needed by the decoder to reconstruct the original data and discards all irrelevant information. The decoder then exploits this information to generate a new image similar to the original one.\"}),/*#__PURE__*/e(\"p\",{children:\"One network discovers the better means of encoding raw data into latent space, and the other network, the decoder, figures out the best ways to transform these latent representations into new relevant content. VAEs, unlike conventional autoencoders, have a characteristic that renders them useful in data generation. Their latent space is constructed in a continuous way, enabling random transformations.\"}),/*#__PURE__*/e(\"h2\",{children:\"Generative AI preparation stages\"}),/*#__PURE__*/e(\"p\",{children:\"The process of training and preparation of different generative AI models will vary depending on the algorithm employed, but the basic steps are as follows:\"}),/*#__PURE__*/e(\"h3\",{children:\"Selection of generative AI model architecture\"}),/*#__PURE__*/e(\"p\",{children:\"Various kinds of models propose a range of options and are appropriate for particular purposes. For example, GANs and diffusion models are better for realistic images generation, transformers show great results in natural language processing and so on. The right kind of model architecture is essential to achieve the desired results. The choice of a suitable type of architecture for a model is decisive for generative AI, as it determines how the model explores and produces relevant data.\"}),/*#__PURE__*/e(\"h3\",{children:\"Data collection and cleaning\"}),/*#__PURE__*/e(\"p\",{children:\"A rather large data set is acquired from a certain area of interest. A massive set of training data is collected or located that holds the type of information that the future model will be required to generate. For instance, it could be a massive collection of texts or millions of images as well as audio recordings, etc. At this stage, experts can also clean the data, split it into smaller units, and convert it into a format that the model will understand.\"}),/*#__PURE__*/e(\"h3\",{children:\"Iterative training and optimization\"}),/*#__PURE__*/e(\"p\",{children:\"The stage of generative AI model training is perhaps one of the most fundamental stages of model preparation. The model is trained to generate novel data on its own. The collected set of training data is submitted for model processing, during which the model analyzes all collected data, looking for features, common elements, and structures to learn the essence of the given data. The reduction of the distinction between the model results and the training data to a minimum enables the model to create synthetic content that is akin to that training data.\"}),/*#__PURE__*/e(\"p\",{children:\"During the training process, various machine learning algorithms are applied for model parameter optimization. It can happen through such methods as stochastic gradient descent (SGD), forward and backward propagation, regularization, and so on.\"}),/*#__PURE__*/e(\"p\",{children:\"To properly train AI models, the model needs to be exposed to training data a considerable number of times. Therefore, training is not a fast process and generally involves multiple stages, referred to as epochs. The more there is data, the greater the number of such stages. Epochs in turn are also partitioned into several iterations. The model undergoes each iteration, thus learning from the data processed in it.\"}),/*#__PURE__*/e(\"h3\",{children:\"Evaluation and deployment\"}),/*#__PURE__*/e(\"p\",{children:\"After training, experts check how well the model has been trained on new test data that was not included in the original training dataset. If the performance of the model is not acceptable, they carry out additional training with modified training parameters. If the model's performance meets the expectations of its creators, the final trained model is applied to create images, text, music, or even program code and other new content.\"}),/*#__PURE__*/e(\"h2\",{children:\"Training data for generative AI models\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI differs from traditional AI algorithms, among other things, in that it does not require data annotation for training. This means that it entails the application of unsupervised and semi-supervised machine learning algorithms. It is interesting to note, however, that there has to be an immense amount of such data, and these massive sets may not always consist of high-quality data.\"}),/*#__PURE__*/e(\"h3\",{children:\"Unsupervised learning\"}),/*#__PURE__*/e(\"p\",{children:\"In this kind of learning, the machine does the learning itself without human intervention and the given data is unlabeled. The point of it is that there are no correct and straightforward answers in the data, the machine has to find patterns and dependencies between the entities. However, even for unsupervised learning, the training input data the model will be trained on often has to be prepared and cleaned.\"}),/*#__PURE__*/e(\"h3\",{children:\"Semi-supervised learning\"}),/*#__PURE__*/e(\"p\",{children:\"Semi-supervised learning is applicable when there is only a small amount of labeled data available or the model is incapable of learning without any labeling. Thus, data scientists possess a data set that consists of two parts: the labeled and the raw data. They feed both parts of the data to machine learning models to train them.\"}),/*#__PURE__*/e(\"h2\",{children:\"Key applications of generative AI\"}),/*#__PURE__*/e(\"h3\",{children:\"Text creation with Large language models\"}),/*#__PURE__*/e(\"p\",{children:\"Advances in deep learning have led to the widespread use of various neural networks for natural language processing (NLP) tasks. Currently, nothing does a better job of composing human-like text than large language models (LLMs). The term large language model usually refers to deep language-based generative AI models having about a billion or more parameters. They cope with a wide range of tasks, as opposed to learning one specific one.\"}),/*#__PURE__*/e(\"p\",{children:\"An LLM consists of a language model composed of a neural network containing a multitude of parameters. It is typically trained on a substantial amount of unlabeled text via unsupervised learning. A language model is essential for understanding and generating content in natural language. Besides having the capability of learning a great deal of the syntax and semantics of the human language, LLMs also demonstrate considerable general knowledge of the world and can recall a wide variety of facts while learning. For instance, the aforementioned GPT is a large language model with a transformer architecture.\"}),/*#__PURE__*/e(\"h3\",{children:\"Image creation\"}),/*#__PURE__*/e(\"p\",{children:\"A recent breakthrough in AI image creation has been possible due to the development of incredibly powerful generative AI models. They may render an image from a text prompt, a source picture, a rough sketch, or references. Certain generative AI models can even complete unfinished artwork. Other services specialize in face generation.\"}),/*#__PURE__*/e(\"p\",{children:\"In the 2020s several popular generative AI interfaces have been created that specialize in creating images and art in a variety of styles that in most cases cannot be distinguished from those created by a person. Such as the Midjourney image-generating tool, for instance, belongs to the class of diffusion networks, i.e. it consists of two networks: one is responsible for text detection, the other for image generation.\"}),/*#__PURE__*/e(\"p\",{children:\"Another image-generating tool is called DALL-E. It generates images based on text requests and knows how to produce versions of pictures guided by examples suggested by the user. It relies on three different networks, each serving its own function. The first one, called CLIP, produces sketches based on the pre-recognized text submitted by the user. The sketch is then converted into a low-resolution image by the GLIDE network. And the third network increases the resolution of the final resulting image in the output.\"}),/*#__PURE__*/e(\"p\",{children:\"Another image generation tool called Stable diffusion is an open-source service unlike the ones mentioned above. Yet it offers high-quality generated pictures as well. Stable Diffusion is similarly built on diffusion models, which means the gradual transformation of noise into the target image.\"}),/*#__PURE__*/e(\"h2\",{children:\"Benefits of Generative AI\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI, when used appropriately and correctly, is a powerful instrument to help produce new insights, find answers to solve existing issues, and discover innovations, all of which may be a tremendous asset in the workplace. There are numerous pros to applying generative AI in business, such as:\"}),/*#__PURE__*/e(\"h3\",{children:\"Faster deliverables, with increased efficiency and productivity\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI helps your employees complete tasks faster and with greater efficiency. AI is capable of working independently or alongside your staff, replacing some routine operations. For example, generative AI can create high-quality document templates, product descriptions, articles, or eye-catching graphic elements and videos faster than a human can. With such a powerful and fast system, multiple content pieces may be generated at the same time.\"}),/*#__PURE__*/e(\"h3\",{children:\"Reducing costs\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI is capable of automating many mundane processes in a company. Also, for example, a company may save funds when creating promotional content and developing program code. This can be achieved through the implementation of professional generative AI tools that are comparatively cheaper than doing the tasks on your own.\"}),/*#__PURE__*/e(\"h3\",{children:\"Competitive advantage\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI helps companies make better and faster decisions. It means that they will be able to, for example, rapidly increase their social media presence. One of the ways to accomplish this is to maximize the amount of content posted with the help of generative AI and publish it earlier than competitors. Such an approach gives a company an edge over its rivals by attracting more attention as well as reaching more customers.\"}),/*#__PURE__*/e(\"h3\",{children:\"Higher originality and lower error rates\"}),/*#__PURE__*/e(\"p\",{children:\"Fewer errors mean less time for revisions and improvements, enabling the generation of more original content and products. Meanwhile, the increase in the quantity of content does not affect the quality and originality of the data, as generative AI always provides unique and lifelike content that can be even further refined using generative AI.\"}),/*#__PURE__*/e(\"h2\",{children:\"Conclusion\"}),/*#__PURE__*/e(\"p\",{children:\"Generative AI is a technology that employs complex generative models that can produce new data, like audio, code, images, text, and video. This type of AI, in contrary to discriminative AI capable of performing tasks like identification and classification, builds new pieces of information using foundation models, which are deep AI models that solve complex problems simultaneously.\"}),/*#__PURE__*/e(\"p\",{children:\"It's safe to say that we currently live in the prime era of generative AI models. Even today AI solutions propose the top-quality results. The future holds the rise of even more advanced AI systems that will be able to generate unique and realistic content faster and with minimal to zero errors. The generative AI tools will become more accessible and more intuitive over time. They will have the power to change almost every aspect of our lives and business.\"})]});\nexport const __FramerMetadata__ = {\"exports\":{\"richText5\":{\"type\":\"variable\",\"annotations\":{\"framerContractVersion\":\"1\"}},\"richText3\":{\"type\":\"variable\",\"annotations\":{\"framerContractVersion\":\"1\"}},\"richText8\":{\"type\":\"variable\",\"annotations\":{\"framerContractVersion\":\"1\"}},\"richText1\":{\"type\":\"variable\",\"annotations\":{\"framerContractVersion\":\"1\"}},\"richText7\":{\"type\":\"variable\",\"annotations\":{\"framerContractVersion\":\"1\"}},\"richText2\":{\"type\":\"variable\",\"annotations\":{\"framerContractVersion\":\"1\"}},\"richText9\":{\"type\":\"variable\",\"annotations\":{\"framerContractVersion\":\"1\"}},\"richText4\":{\"type\":\"variable\",\"annotations\":{\"framerContractVersion\":\"1\"}},\"richText6\":{\"type\":\"variable\",\"annotations\":{\"framerContractVersion\":\"1\"}},\"richText\":{\"type\":\"variable\",\"annotations\":{\"framerContractVersion\":\"1\"}},\"__FramerMetadata__\":{\"type\":\"variable\"}}}"],
  "mappings": "+LAAsJ,IAAMA,EAAsBC,EAAIC,EAAS,CAAC,SAAS,CAAcC,EAAE,IAAI,CAAC,SAAS,wQAAwQ,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,+IAAuJE,EAAEC,EAAE,CAAC,KAAK,uCAAuC,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,UAAU,CAAC,EAAE,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,6DAA6D,CAAC,CAAC,CAAC,EAAE,+JAA+J,CAAC,CAAC,EAAeF,EAAE,MAAM,CAAC,IAAI,2JAA2J,UAAU,eAAe,OAAO,MAAM,IAAI,uEAAuE,OAAO,qWAAqW,MAAM,CAAC,YAAY,aAAa,EAAE,MAAM,MAAM,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,qYAA4W,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,kBAAkB,CAAC,EAAeA,EAAE,MAAM,CAAC,IAAI,mBAAmB,UAAU,eAAe,OAAO,MAAM,IAAI,wEAAwE,OAAO,yWAAyW,MAAM,CAAC,YAAY,aAAa,EAAE,MAAM,MAAM,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,miBAA2iBE,EAAEC,EAAE,CAAC,KAAK,iCAAiC,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,UAAU,CAAC,EAAE,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,sLAAsL,CAAC,CAAC,CAAC,EAAE,SAAI,CAAC,CAAC,EAAeJ,EAAE,IAAI,CAAC,SAAS,CAAC,+BAA4CE,EAAEC,EAAE,CAAC,KAAK,iCAAiC,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,UAAU,CAAC,EAAE,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,QAAQ,CAAC,CAAC,CAAC,EAAE,KAAkBF,EAAEC,EAAE,CAAC,KAAK,yEAAyE,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,UAAU,CAAC,EAAE,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,SAAS,CAAC,CAAC,CAAC,EAAE,QAAqBF,EAAEC,EAAE,CAAC,KAAK,qGAAqG,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,UAAU,CAAC,EAAE,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,UAAU,CAAC,CAAC,CAAC,EAAE,8qBAA8qB,CAAC,CAAC,EAAeF,EAAE,KAAK,CAAC,SAAS,2BAA2B,CAAC,EAAeA,EAAE,MAAM,CAAC,IAAI,4BAA4B,UAAU,eAAe,OAAO,MAAM,IAAI,uEAAuE,OAAO,qWAAqW,MAAM,CAAC,YAAY,aAAa,EAAE,MAAM,MAAM,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,4ZAA6Y,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,03BAA03B,CAAC,EAAeA,EAAE,MAAM,CAAC,IAAI,iIAAiI,UAAU,eAAe,OAAO,MAAM,IAAI,uEAAuE,OAAO,qWAAqW,MAAM,CAAC,YAAY,aAAa,EAAE,MAAM,MAAM,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,qFAAqF,CAAC,EAAeA,EAAE,MAAM,CAAC,IAAI,sFAAsF,UAAU,eAAe,OAAO,OAAO,IAAI,wEAAwE,OAAO,2QAA2Q,MAAM,CAAC,YAAY,aAAa,EAAE,MAAM,MAAM,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,0aAA0a,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,6BAA6B,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,mBAAgCE,EAAEC,EAAE,CAAC,KAAK,+CAA+C,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,UAAU,CAAC,EAAE,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,4BAA4B,CAAC,CAAC,CAAC,EAAE,QAAqBF,EAAEC,EAAE,CAAC,KAAK,uCAAuC,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,UAAU,CAAC,EAAE,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,yBAAyB,CAAC,CAAC,CAAC,EAAE,+gBAA0gB,CAAC,CAAC,EAAeJ,EAAE,IAAI,CAAC,SAAS,CAAC,4DAAyEE,EAAEC,EAAE,CAAC,KAAK,uGAAuG,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,UAAU,CAAC,EAAE,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,6BAA6B,CAAC,CAAC,CAAC,EAAE,KAAkBF,EAAEC,EAAE,CAAC,KAAK,qCAAqC,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,UAAU,CAAC,EAAE,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,uBAAuB,CAAC,CAAC,CAAC,EAAE,KAAkBF,EAAEC,EAAE,CAAC,KAAK,8CAA8C,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,UAAU,CAAC,EAAE,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,eAAe,CAAC,CAAC,CAAC,EAAE,KAAkBF,EAAEC,EAAE,CAAC,KAAK,2EAA2E,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,UAAU,CAAC,EAAE,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,0CAA0C,CAAC,CAAC,CAAC,EAAE,wpBAAmpB,CAAC,CAAC,EAAeJ,EAAE,IAAI,CAAC,SAAS,CAAC,2KAAwLE,EAAEC,EAAE,CAAC,KAAK,uGAAuG,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,UAAU,CAAC,EAAE,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,eAAe,CAAC,CAAC,CAAC,EAAE,2HAAwIF,EAAEC,EAAE,CAAC,KAAK,uCAAuC,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,UAAU,CAAC,EAAE,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,uCAAuC,CAAC,CAAC,CAAC,EAAE,sBAAmCF,EAAEC,EAAE,CAAC,KAAK,6MAA6M,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,UAAU,CAAC,EAAE,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,qCAAqC,CAAC,CAAC,CAAC,EAAE,0IAA0I,CAAC,CAAC,EAAeF,EAAE,KAAK,CAAC,SAAS,cAAc,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,iTAA8TE,EAAEC,EAAE,CAAC,KAAK,uCAAuC,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,UAAU,CAAC,EAAE,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,gEAAgE,CAAC,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,mPAAmP,CAAC,EAAeF,EAAE,KAAK,CAAC,MAAM,CAAC,qBAAqB,OAAO,0BAA0B,QAAQ,sBAAsB,kBAAkB,6BAA6B,MAAM,0BAA0B,MAAM,EAAE,SAAS,CAAcE,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,qDAAqD,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAEC,EAAE,CAAC,KAAK,uCAAuC,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,UAAU,CAAC,EAAE,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,8CAA8C,CAAC,CAAC,CAAC,EAAE,0BAA0B,CAAC,CAAC,CAAC,CAAC,EAAeF,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,6HAAwH,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,4DAA4D,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,2FAA2F,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,eAAe,CAAC,EAAeF,EAAE,KAAK,CAAC,MAAM,CAAC,qBAAqB,OAAO,0BAA0B,QAAQ,sBAAsB,kBAAkB,6BAA6B,MAAM,0BAA0B,MAAM,EAAE,SAAS,CAAcE,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,mGAAmG,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,0FAA0F,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,8LAAyL,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeG,EAAuBL,EAAIC,EAAS,CAAC,SAAS,CAAcC,EAAE,KAAK,CAAC,SAAS,YAAY,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,4FAAyGE,EAAE,SAAS,CAAC,SAAS,MAAM,CAAC,EAAE,KAAkBA,EAAE,SAAS,CAAC,SAAS,OAAO,CAAC,EAAE,SAAsBA,EAAE,SAAS,CAAC,SAAS,QAAQ,CAAC,EAAE,YAAY,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,qCAAqC,CAAC,EAAeF,EAAE,KAAK,CAAC,SAAS,CAAcE,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,6CAA6C,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,gBAAgB,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,qDAAqD,CAAC,EAAeF,EAAE,KAAK,CAAC,SAAS,CAAcE,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,6LAAmL,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,uFAAuF,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,yIAAyI,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,mLAAmL,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,UAAU,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,mUAA8T,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,gWAAgW,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,yHAAyH,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,yIAAyI,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,yBAAyB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,0IAA0I,CAAC,EAAeF,EAAE,KAAK,CAAC,SAAS,CAAcE,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,wIAAwI,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,gLAAgL,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,0DAA0D,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,ySAAyS,CAAC,CAAC,CAAC,CAAC,EAAeI,EAAuBN,EAAIC,EAAS,CAAC,SAAS,CAAcC,EAAE,KAAK,CAAC,SAAS,kBAAkB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,0QAA0Q,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,WAAW,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,oOAAqN,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,2SAA2S,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,UAAU,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,iGAA4F,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,kCAAkC,CAAC,EAAeF,EAAE,KAAK,CAAC,MAAM,CAAC,qBAAqB,OAAO,0BAA0B,QAAQ,sBAAsB,kBAAkB,6BAA6B,MAAM,0BAA0B,OAAO,oBAAoB,OAAO,EAAE,SAAS,CAAcE,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,+EAA+E,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,0DAA0D,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,2IAAiI,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,uFAAuF,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,mFAAmF,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,oFAAoF,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,8CAAyC,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,iCAAiC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,iOAAiO,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,mCAAmC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,kRAA6Q,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,qDAA2C,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,gbAA2a,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,sFAAsF,CAAC,EAAeA,EAAE,MAAM,CAAC,IAAI,8EAA8E,UAAU,eAAe,OAAO,OAAO,IAAI,wEAAwE,OAAO,wcAAwc,MAAM,CAAC,YAAY,aAAa,EAAE,MAAM,MAAM,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,8EAA8E,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,4BAA4B,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,6HAAmH,CAAC,EAAeF,EAAE,KAAK,CAAC,MAAM,CAAC,qBAAqB,OAAO,0BAA0B,QAAQ,sBAAsB,kBAAkB,6BAA6B,MAAM,0BAA0B,MAAM,EAAE,SAAS,CAAcE,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,mHAAmH,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,0FAA0F,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,0FAA0F,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,qJAAqJ,CAAC,EAAeA,EAAE,MAAM,CAAC,IAAI,kFAAkF,UAAU,eAAe,OAAO,MAAM,IAAI,sEAAsE,OAAO,iWAAiW,MAAM,CAAC,YAAY,aAAa,EAAE,MAAM,MAAM,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,kFAAkF,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,iBAAiB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,qOAAqO,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,+GAA+G,CAAC,CAAC,CAAC,CAAC,EAAeK,EAAuBP,EAAIC,EAAS,CAAC,SAAS,CAAcC,EAAE,IAAI,CAAC,SAAS,qTAAqT,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,oWAAoW,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,sDAAsD,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,+GAA+G,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,uVAAuV,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,mfAAmf,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,wBAAwB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,wOAAwO,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,oJAAoJ,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,meAAme,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,6OAA6O,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,oHAAoH,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,qBAAqB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,4UAA4U,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,oOAAoO,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,MAAM,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,uPAAuP,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,uOAAuO,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,QAAQ,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,qOAAqO,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,OAAO,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,oLAAoL,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,OAAO,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,2MAA2M,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,WAAW,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,6fAA6f,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,QAAQ,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,uPAAuP,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,8BAA8B,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,scAAsc,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,wcAAwc,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,oCAAoC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,4OAA4O,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,uWAAuW,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,gUAAgU,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,6VAA6V,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,sBAAsB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,gEAAgE,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,wCAAwC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,yVAAyV,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,0bAA0b,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,iCAAiC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,gXAAgX,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,0BAA0B,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,2dAA2d,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,4hBAA4hB,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,kBAAkB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,ydAAyd,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,wCAAwC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,iGAAiG,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,sBAAsB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,iUAAiU,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,+BAA+B,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,wMAAwM,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,sBAAsB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,sRAAsR,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,gBAAgB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,kRAAkR,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,0BAA0B,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,oNAAoN,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,mBAAmB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,kWAAkW,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,mCAAmC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,qcAAqc,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,0aAA0a,CAAC,CAAC,CAAC,CAAC,EAAeM,EAAuBR,EAAIC,EAAS,CAAC,SAAS,CAAcC,EAAE,IAAI,CAAC,SAAS,4qBAAkqB,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,yCAAyC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,0xBAA4vB,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,mCAAmC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,mgBAA8f,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,sBAAsB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,gwBAAgwB,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,6DAA6D,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,sHAAmIE,EAAEC,EAAE,CAAC,KAAK,2CAA2C,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,UAAU,CAAC,EAAE,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,aAAa,CAAC,CAAC,CAAC,EAAE,0TAA0T,CAAC,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,+TAA+T,CAAC,CAAC,CAAC,CAAC,EAAeO,EAAuBT,EAAIC,EAAS,CAAC,SAAS,CAAcC,EAAE,IAAI,CAAC,SAAS,gzBAAuxB,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,wBAAwB,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,m3BAAg4BE,EAAE,KAAK,CAAC,SAAS,WAAW,CAAC,EAAE,QAAqBA,EAAE,KAAK,CAAC,SAAS,cAAc,CAAC,EAAE,6XAAgYA,EAAEC,EAAE,CAAC,KAAK,qBAAqB,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,UAAU,CAAC,EAAE,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,WAAW,CAAC,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC,EAAeF,EAAE,KAAK,CAAC,SAAS,8CAAyC,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,kDAA+DE,EAAEC,EAAE,CAAC,KAAK,6BAA6B,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,UAAU,CAAC,EAAE,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,YAAY,CAAC,CAAC,CAAC,EAAE,yrBAAyrB,CAAC,CAAC,EAAeF,EAAE,KAAK,CAAC,SAAS,mDAAmD,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,2WAAwXE,EAAE,KAAK,CAAC,SAAS,6BAAmB,CAAC,EAAE,OAAoBA,EAAE,KAAK,CAAC,SAAS,gCAAsB,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,2XAAiX,CAAC,EAAeA,EAAE,MAAM,CAAC,IAAI,WAAW,UAAU,eAAe,OAAO,MAAM,IAAI,wEAAwE,OAAO,yWAAyW,MAAM,CAAC,YAAY,aAAa,EAAE,MAAM,MAAM,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,6KAA6K,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,6mBAAmmB,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,mEAA+C,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,yKAAsLE,EAAE,SAAS,CAAC,SAAS,WAAW,CAAC,EAAE,OAAoBA,EAAE,SAAS,CAAC,SAAS,eAAe,CAAC,EAAE,wLAAwL,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,MAAM,CAAC,qBAAqB,OAAO,0BAA0B,QAAQ,sBAAsB,kBAAkB,6BAA6B,MAAM,0BAA0B,OAAO,oBAAoB,OAAO,EAAE,SAAsBA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,sBAAsB,CAAC,EAAE,KAAkBA,EAAE,KAAK,CAAC,SAAS,uJAAuJ,CAAC,EAAE,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,UAAU,CAAC,EAAE,KAAkBA,EAAE,KAAK,CAAC,SAAS,oDAA0C,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,oFAA0E,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,uFAA6E,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,6DAAmD,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,sGAA4F,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,sHAA4G,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,MAAM,IAAI,MAAM,CAAC,qBAAqB,OAAO,0BAA0B,QAAQ,sBAAsB,kBAAkB,6BAA6B,MAAM,0BAA0B,OAAO,oBAAoB,OAAO,EAAE,SAAsBA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,kBAAkB,CAAC,EAAE,oHAAoH,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,UAAU,CAAC,EAAE,KAAkBA,EAAE,KAAK,CAAC,SAAS,iGAAuF,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,oCAA0B,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,uEAA6D,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,qDAA2C,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,MAAM,IAAI,MAAM,CAAC,qBAAqB,OAAO,0BAA0B,QAAQ,sBAAsB,kBAAkB,6BAA6B,MAAM,0BAA0B,OAAO,oBAAoB,OAAO,EAAE,SAAsBA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,kBAAkB,CAAC,EAAE,KAAkBA,EAAE,KAAK,CAAC,SAAS,6GAA6G,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,UAAU,CAAC,EAAE,KAAkBA,EAAE,KAAK,CAAC,SAAS,wFAA8E,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,+FAAqF,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,4EAAkE,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,wFAA8E,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,MAAM,IAAI,MAAM,CAAC,qBAAqB,OAAO,0BAA0B,QAAQ,sBAAsB,kBAAkB,6BAA6B,MAAM,0BAA0B,OAAO,oBAAoB,OAAO,EAAE,SAAsBA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,WAAW,CAAC,EAAE,KAAkBA,EAAE,KAAK,CAAC,SAAS,kGAAkG,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,UAAU,CAAC,EAAE,KAAkBA,EAAE,KAAK,CAAC,SAAS,+IAA2H,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,kFAA8D,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,iHAAkG,CAAC,CAAC,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,oCAAiDE,EAAE,SAAS,CAAC,SAAS,2BAAiB,CAAC,EAAE,sDAAsD,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,MAAM,CAAC,qBAAqB,OAAO,0BAA0B,QAAQ,sBAAsB,kBAAkB,6BAA6B,MAAM,0BAA0B,OAAO,oBAAoB,OAAO,EAAE,SAAsBA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,aAAa,CAAC,EAAE,KAAkBA,EAAE,KAAK,CAAC,SAAS,yFAAyF,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,UAAU,CAAC,EAAE,KAAkBA,EAAE,KAAK,CAAC,SAAS,iCAAuB,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,8BAAoB,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,gDAAsC,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,iDAA6B,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,yCAA0B,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,8BAAoB,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,MAAM,IAAI,MAAM,CAAC,qBAAqB,OAAO,0BAA0B,QAAQ,sBAAsB,kBAAkB,6BAA6B,MAAM,0BAA0B,OAAO,oBAAoB,OAAO,EAAE,SAAsBA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,mBAAmB,CAAC,EAAE,KAAkBA,EAAE,KAAK,CAAC,SAAS,sGAAsG,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,UAAU,CAAC,EAAE,KAAkBA,EAAE,KAAK,CAAC,SAAS,kEAAmD,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,qFAAsE,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,6EAAmE,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,0FAAgF,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,MAAM,IAAI,MAAM,CAAC,qBAAqB,OAAO,0BAA0B,QAAQ,sBAAsB,kBAAkB,6BAA6B,MAAM,0BAA0B,OAAO,oBAAoB,OAAO,EAAE,SAAsBA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,uBAAuB,CAAC,EAAE,KAAkBA,EAAE,KAAK,CAAC,SAAS,oHAA+G,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,UAAU,CAAC,EAAE,KAAkBA,EAAE,KAAK,CAAC,SAAS,mEAAyD,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,8EAA+D,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,uEAA6D,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,sEAAkD,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,yDAA+C,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,MAAM,IAAI,MAAM,CAAC,qBAAqB,OAAO,0BAA0B,QAAQ,sBAAsB,kBAAkB,6BAA6B,MAAM,0BAA0B,OAAO,oBAAoB,OAAO,EAAE,SAAsBA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,qBAAqB,CAAC,EAAE,KAAkBA,EAAE,KAAK,CAAC,SAAS,mGAAmG,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,UAAU,CAAC,EAAE,KAAkBA,EAAE,KAAK,CAAC,SAAS,oEAA0D,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,yDAA+C,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,0EAA2D,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,yEAA+D,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,MAAM,IAAI,MAAM,CAAC,qBAAqB,OAAO,0BAA0B,QAAQ,sBAAsB,kBAAkB,6BAA6B,MAAM,0BAA0B,OAAO,oBAAoB,OAAO,EAAE,SAAsBA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,aAAa,CAAC,EAAE,KAAkBA,EAAE,KAAK,CAAC,SAAS,8FAAyF,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,UAAU,CAAC,EAAE,KAAkBA,EAAE,KAAK,CAAC,SAAS,wCAA8B,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,yCAA+B,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,oEAA0D,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,oEAA0D,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,kDAAmC,CAAC,CAAC,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,4BAAyCE,EAAE,SAAS,CAAC,SAAS,uBAAa,CAAC,EAAE,QAAqBA,EAAE,SAAS,CAAC,SAAS,2BAAiB,CAAC,EAAE,8tBAA8tB,CAAC,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,0DAAuEE,EAAE,SAAS,CAAC,SAAS,uBAAa,CAAC,EAAE,kXAAkX,CAAC,CAAC,EAAeF,EAAE,KAAK,CAAC,SAAS,CAAcE,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAC,4BAAyCE,EAAE,KAAK,CAAC,SAAS,mDAAmD,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAC,+BAA4CE,EAAE,KAAK,CAAC,SAAS,8DAA8D,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAC,oBAAiCE,EAAE,KAAK,CAAC,SAAS,uEAAuE,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAC,eAA4BE,EAAE,KAAK,CAAC,SAAS,2EAA2E,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAC,eAA4BE,EAAE,KAAK,CAAC,SAAS,8EAA8E,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,MAAM,CAAC,IAAI,WAAW,UAAU,eAAe,OAAO,MAAM,IAAI,sEAAsE,OAAO,oQAAoQ,MAAM,CAAC,YAAY,YAAY,EAAE,MAAM,KAAK,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,mNAAmN,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,4BAA4B,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,mSAAmS,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,4pBAAupB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,uiBAAkiB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,4JAA4J,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,aAAa,CAAC,EAAeF,EAAE,KAAK,CAAC,SAAS,CAAcE,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAC,+GAAkHE,EAAE,KAAK,CAAC,SAAS,+DAA+D,CAAC,EAAE,2BAAsB,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,wIAAwI,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,iPAAuO,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAEC,EAAE,CAAC,KAAK,8GAA8G,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,UAAU,CAAC,EAAE,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,6GAA6G,CAAC,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeM,EAAuBV,EAAIC,EAAS,CAAC,SAAS,CAAcD,EAAE,IAAI,CAAC,SAAS,CAAC,kPAA+PE,EAAE,KAAK,CAAC,SAAS,iBAAiB,CAAC,EAAE,qeAAqe,CAAC,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,8SAA2TE,EAAE,KAAK,CAAC,SAAS,0CAAqC,CAAC,EAAE,2MAAmNA,EAAE,KAAK,CAAC,SAAS,2DAAiD,CAAC,EAAE,6JAAwJ,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,iCAAiC,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,8ZAAuZE,EAAE,KAAK,CAAC,SAAS,eAAe,CAAC,EAAE,0TAA4R,CAAC,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,srBAAmsBE,EAAEC,EAAE,CAAC,KAAK,qBAAqB,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,UAAU,CAAC,EAAE,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,WAAW,CAAC,CAAC,CAAC,EAAE,4HAA+HF,EAAEC,EAAE,CAAC,KAAK,2BAA2B,YAAY,GAAG,OAAO,YAAY,aAAa,GAAG,UAAU,CAAC,EAAE,QAAQ,oBAAoB,aAAa,GAAG,SAAsBD,EAAEE,EAAE,EAAE,CAAC,SAAS,UAAU,CAAC,CAAC,CAAC,EAAE,mFAAyE,CAAC,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,6YAA6Y,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,klCAAwkC,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,qCAAqC,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,ygBAA4gBE,EAAE,KAAK,CAAC,SAAS,YAAY,CAAC,EAAE,OAAO,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,2mBAAsmB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,2cAA2c,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,qCAAqC,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,8MAA2NE,EAAE,SAAS,CAAC,SAAS,iBAAiB,CAAC,EAAE,sQAAyQA,EAAE,SAAS,CAAC,SAAS,kBAAkB,CAAC,EAAE,iRAA8RA,EAAE,SAAS,CAAC,SAAS,YAAY,CAAC,EAAE,yNAA4NA,EAAE,SAAS,CAAC,SAAS,aAAa,CAAC,EAAE,yKAAyK,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,4CAA4C,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,4XAA4X,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,6ZAA6Z,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,8aAA2bE,EAAE,KAAK,CAAC,SAAS,sBAAsB,CAAC,EAAE,6fAA6f,CAAC,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,+ZAA4aE,EAAE,KAAK,CAAC,SAAS,6CAAmC,CAAC,EAAE,0TAA0T,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,6BAA6B,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,whBAAwhB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,wXAAwX,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,ysBAAysB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,2dAA2d,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,saAAia,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,0EAA0E,CAAC,EAAeF,EAAE,IAAI,CAAC,SAAS,CAAC,0rBAAwrBE,EAAE,SAAS,CAAC,SAAS,YAAY,CAAC,EAAE,4LAAwK,CAAC,CAAC,EAAeA,EAAE,MAAM,CAAC,IAAI,mMAAyL,UAAU,eAAe,OAAO,OAAO,IAAI,wEAAwE,OAAO,2QAA2Q,MAAM,CAAC,YAAY,aAAa,EAAE,MAAM,MAAM,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,kMAAwL,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,yyBAA+xB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,yaAAoa,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,YAAY,CAAC,EAAeF,EAAE,KAAK,CAAC,MAAM,CAAC,qBAAqB,OAAO,0BAA0B,QAAQ,sBAAsB,kBAAkB,6BAA6B,MAAM,0BAA0B,OAAO,oBAAoB,OAAO,EAAE,SAAS,CAAcE,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAC,0IAAuJE,EAAE,KAAK,CAAC,SAAS,iEAAiE,CAAC,EAAE,4BAAuB,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAC,2JAAwKE,EAAE,KAAK,CAAC,SAAS,2EAA2E,CAAC,EAAE,wBAAmB,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAC,mCAAgDE,EAAE,KAAK,CAAC,SAAS,8EAA8E,CAAC,EAAE,sBAAsB,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAC,+GAA4HE,EAAE,KAAK,CAAC,SAAS,iHAAiH,CAAC,EAAE,cAAc,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAC,kNAAqNE,EAAE,KAAK,CAAC,SAAS,wFAAwF,CAAC,EAAE,2BAAsB,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAC,mIAAgJE,EAAE,KAAK,CAAC,SAAS,6DAA6D,CAAC,EAAE,oBAAe,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAC,iHAA8HE,EAAE,KAAK,CAAC,SAAS,wFAAwF,CAAC,EAAE,oBAAe,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAC,8HAA2IE,EAAE,KAAK,CAAC,SAAS,+EAA+E,CAAC,EAAE,wBAAmB,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAC,qNAAkOE,EAAE,KAAK,CAAC,SAAS,iCAAiC,CAAC,EAAE,GAAG,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAC,kJAA+JE,EAAE,KAAK,CAAC,SAAS,sBAAsB,CAAC,EAAE,8BAAyB,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAC,oKAAiLE,EAAE,KAAK,CAAC,SAAS,iCAAiC,CAAC,EAAE,GAAG,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAC,2JAAwKE,EAAE,KAAK,CAAC,SAAS,8EAA8E,CAAC,EAAE,mBAAc,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAC,8IAA2JE,EAAE,KAAK,CAAC,SAAS,2EAA2E,CAAC,EAAE,4BAAuB,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAC,2GAAwHE,EAAE,KAAK,CAAC,SAAS,wEAAwE,CAAC,EAAE,sBAAiB,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAC,uHAAoIE,EAAE,KAAK,CAAC,SAAS,iCAAiC,CAAC,EAAE,GAAG,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAC,+KAA4LE,EAAE,KAAK,CAAC,SAAS,iCAAiC,CAAC,EAAE,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeS,EAAuBX,EAAIC,EAAS,CAAC,SAAS,CAAcC,EAAE,KAAK,CAAC,SAAS,mBAAmB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,2TAAsT,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,WAAW,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,8VAA+U,CAAC,EAAeA,EAAE,MAAM,CAAC,IAAI,YAAY,UAAU,eAAe,OAAO,MAAM,IAAI,uEAAuE,OAAO,qWAAqW,MAAM,CAAC,YAAY,aAAa,EAAE,MAAM,MAAM,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,UAAU,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,mIAA8H,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,iEAAiE,CAAC,EAAeF,EAAE,KAAK,CAAC,MAAM,CAAC,qBAAqB,OAAO,0BAA0B,QAAQ,sBAAsB,kBAAkB,6BAA6B,MAAM,0BAA0B,MAAM,EAAE,SAAS,CAAcE,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,kBAAkB,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,WAAW,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,gBAAgB,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,MAAM,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,kBAAkB,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,eAAe,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,iBAAiB,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,qPAAgP,CAAC,EAAeA,EAAE,MAAM,CAAC,IAAI,WAAW,UAAU,eAAe,OAAO,OAAO,IAAI,wEAAwE,OAAO,yWAAyW,MAAM,CAAC,YAAY,aAAa,EAAE,MAAM,MAAM,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,wQAAwQ,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,iBAAiB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,2WAAsW,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,yIAAyI,CAAC,CAAC,CAAC,CAAC,EAAeU,EAAuBZ,EAAIC,EAAS,CAAC,SAAS,CAAcC,EAAE,IAAI,CAAC,SAAS,iiBAAiiB,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,yBAAyB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,4UAA4U,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,ueAAue,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,4UAA4U,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,gXAAgX,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,+WAAqW,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,8QAA8Q,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,kCAAkC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,qeAAqe,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,kPAAkP,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,wSAAwS,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,kaAAka,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,+dAA+d,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,+dAA+d,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,miBAAmiB,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,yBAAyB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,+kBAA+kB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,qQAAqQ,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,8SAA8S,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,yCAAyC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,gkBAAgkB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,gYAAgY,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,4XAA4X,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,kTAAkT,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,oCAAoC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,6PAA6P,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,wRAAwR,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,wXAAwX,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,0NAA0N,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,gQAAgQ,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,qTAAqT,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,cAAc,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,sMAAsM,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,OAAO,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,ycAAyc,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,kCAAkC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,kdAAkd,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,gEAAgE,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,qTAAqT,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,sCAAsC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,sSAAsS,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,ySAAyS,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,YAAY,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,+UAA+U,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,yeAAye,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,sbAAsb,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,cAAc,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,KAAK,CAAC,SAAS,4fAA4f,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeW,EAAuBb,EAAIC,EAAS,CAAC,SAAS,CAAcC,EAAE,IAAI,CAAC,SAAS,kUAAkU,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,sTAAsT,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,kUAAkU,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,yBAAyB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,uMAAuM,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,sVAAsV,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,mQAAmQ,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,yfAAyf,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,yLAAyL,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,kCAAkC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,6hBAA6hB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,6QAA6Q,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,8BAA8B,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,sPAAsP,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,+BAA+B,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,0DAA0D,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,SAAS,CAAC,SAAS,wCAAwC,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,kcAAkc,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,kQAAkQ,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,yXAAyX,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,kIAAkI,CAAC,EAAeF,EAAE,KAAK,CAAC,MAAM,CAAC,qBAAqB,OAAO,0BAA0B,QAAQ,sBAAsB,kBAAkB,6BAA6B,MAAM,0BAA0B,OAAO,oBAAoB,OAAO,EAAE,SAAS,CAAcE,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,8IAA8I,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,uDAAuD,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,2KAA2K,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,mGAAmG,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,sJAAsJ,CAAC,EAAeF,EAAE,KAAK,CAAC,MAAM,CAAC,qBAAqB,OAAO,0BAA0B,QAAQ,sBAAsB,kBAAkB,6BAA6B,MAAM,0BAA0B,MAAM,EAAE,SAAS,CAAcE,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,+HAA+H,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,kEAAkE,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBA,EAAE,IAAI,CAAC,SAAS,kLAAkL,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,SAAS,CAAC,SAAS,0BAA0B,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,kTAAkT,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,4UAA4U,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,+6BAA+6B,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAsBA,EAAE,SAAS,CAAC,SAAS,kBAAkB,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,8VAA8V,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,0DAA0D,CAAC,EAAeF,EAAE,KAAK,CAAC,MAAM,CAAC,qBAAqB,OAAO,0BAA0B,QAAQ,sBAAsB,kBAAkB,6BAA6B,MAAM,0BAA0B,OAAO,oBAAoB,OAAO,EAAE,SAAS,CAAcE,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,mBAAmB,CAAC,EAAE,gKAAgK,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,KAAK,CAAC,kBAAkB,IAAI,SAAsBF,EAAE,IAAI,CAAC,SAAS,CAAcE,EAAE,SAAS,CAAC,SAAS,mBAAmB,CAAC,EAAE,6NAA6N,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,yRAAyR,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,waAAwa,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,iCAAiC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,icAAic,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,sZAAsZ,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,kCAAkC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,8JAA8J,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,+CAA+C,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,6eAA6e,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,8BAA8B,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,8cAA8c,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,qCAAqC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,+iBAA+iB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,sPAAsP,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,maAAma,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,2BAA2B,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,sbAAsb,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,wCAAwC,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,8YAA8Y,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,uBAAuB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,8ZAA8Z,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,0BAA0B,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,8UAA8U,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,mCAAmC,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,0CAA0C,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,0bAA0b,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,omBAAomB,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,gBAAgB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,iVAAiV,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,uaAAua,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,0gBAA0gB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,ySAAyS,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,2BAA2B,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,gTAAgT,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,iEAAiE,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,ucAAuc,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,gBAAgB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,6UAA6U,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,uBAAuB,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,ibAAib,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,0CAA0C,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,2VAA2V,CAAC,EAAeA,EAAE,KAAK,CAAC,SAAS,YAAY,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,iYAAiY,CAAC,EAAeA,EAAE,IAAI,CAAC,SAAS,8cAA8c,CAAC,CAAC,CAAC,CAAC,EAC9nuIY,EAAqB,CAAC,QAAU,CAAC,UAAY,CAAC,KAAO,WAAW,YAAc,CAAC,sBAAwB,GAAG,CAAC,EAAE,UAAY,CAAC,KAAO,WAAW,YAAc,CAAC,sBAAwB,GAAG,CAAC,EAAE,UAAY,CAAC,KAAO,WAAW,YAAc,CAAC,sBAAwB,GAAG,CAAC,EAAE,UAAY,CAAC,KAAO,WAAW,YAAc,CAAC,sBAAwB,GAAG,CAAC,EAAE,UAAY,CAAC,KAAO,WAAW,YAAc,CAAC,sBAAwB,GAAG,CAAC,EAAE,UAAY,CAAC,KAAO,WAAW,YAAc,CAAC,sBAAwB,GAAG,CAAC,EAAE,UAAY,CAAC,KAAO,WAAW,YAAc,CAAC,sBAAwB,GAAG,CAAC,EAAE,UAAY,CAAC,KAAO,WAAW,YAAc,CAAC,sBAAwB,GAAG,CAAC,EAAE,UAAY,CAAC,KAAO,WAAW,YAAc,CAAC,sBAAwB,GAAG,CAAC,EAAE,SAAW,CAAC,KAAO,WAAW,YAAc,CAAC,sBAAwB,GAAG,CAAC,EAAE,mBAAqB,CAAC,KAAO,UAAU,CAAC,CAAC",
  "names": ["richText", "u", "x", "p", "Link", "motion", "richText1", "richText2", "richText3", "richText4", "richText5", "richText6", "richText7", "richText8", "richText9", "__FramerMetadata__"]
}