Workshop/3_Summarization.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Summarization\n",
    "A common use case is wanting to summarize long documents. This naturally runs into the context window limitations. Unlike in question-answering, you can't just do some semantic search hacks to only select the chunks of text most relevant to the question (because, in this case, there is no particular question - you want to summarize everything). So what do you do then?\n",
    "\n",
    "The most common way around this is to split the documents into chunks and then do summarization in a recursive manner. By this we mean you first summarize each chunk by itself, then you group the summaries into chunks and summarize each chunk of summaries, and continue doing that until only one is left."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Set Environment variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os  \n",
    "import json  \n",
    "import openai\n",
    "from Utilities.envVars import *\n",
    "\n",
    "# Set Search Service endpoint, index name, and API key from environment variables\n",
    "indexName = SearchIndex\n",
    "\n",
    "# Set OpenAI API key and endpoint\n",
    "openai.api_type = \"azure\"\n",
    "openai.api_version = OpenAiVersion\n",
    "openai_api_key = OpenAiKey\n",
    "assert openai_api_key, \"ERROR: Azure OpenAI Key is missing\"\n",
    "openai.api_key = openai_api_key\n",
    "openAiEndPoint = f\"https://{OpenAiService}.openai.azure.com\"\n",
    "assert openAiEndPoint, \"ERROR: Azure OpenAI Endpoint is missing\"\n",
    "assert \"openai.azure.com\" in openAiEndPoint.lower(), \"ERROR: Azure OpenAI Endpoint should be in the form: \\n\\n\\t<your unique endpoint identifier>.openai.azure.com\"\n",
    "openai.api_base = openAiEndPoint\n",
    "davincimodel = OpenAiDavinci\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Summarize the document/PDF (instead of getting that data from Vector store or document reposit)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import required libraries\n",
    "# Import required libraries\n",
    "from langchain.llms.openai import AzureOpenAI, OpenAI\n",
    "from langchain.embeddings.openai import OpenAIEmbeddings\n",
    "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
    "from langchain.document_loaders import (\n",
    "    PDFMinerLoader,\n",
    "    UnstructuredFileLoader,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set the file name and the namespace for the index\n",
    "fileName = \"Fabric Get Started.pdf\"\n",
    "fabricGetStartedPath = \"Data/PDF/\" + fileName\n",
    "# Load the PDF with Document Loader available from Langchain\n",
    "loader = PDFMinerLoader(fabricGetStartedPath)\n",
    "rawDocs = loader.load()\n",
    "# Set the source \n",
    "for doc in rawDocs:\n",
    "    doc.metadata['source'] = fabricGetStartedPath\n",
    "\n",
    "textSplitter = RecursiveCharacterTextSplitter(chunk_size=1500, chunk_overlap=0)\n",
    "docs = textSplitter.split_documents(rawDocs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of documents chunks generated from PDF :  58\n"
     ]
    }
   ],
   "source": [
    "print(\"Number of documents chunks generated from PDF : \", len(docs))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.chains.qa_with_sources import load_qa_with_sources_chain\n",
    "from langchain.llms.openai import AzureOpenAI, OpenAI\n",
    "from langchain.embeddings.openai import OpenAIEmbeddings\n",
    "from langchain.docstore.document import Document\n",
    "from langchain.prompts import PromptTemplate\n",
    "from IPython.display import display, HTML\n",
    "from langchain.chains.summarize import load_summarize_chain\n",
    "\n",
    "# Flexibility to change the call to OpenAI or Azure OpenAI\n",
    "embeddingModelType = \"azureopenai\"\n",
    "temperature = 0.3\n",
    "tokenLength = 1000\n",
    "\n",
    "if (embeddingModelType == 'azureopenai'):\n",
    "    openai.api_type = \"azure\"\n",
    "    openai.api_key = OpenAiKey\n",
    "    openai.api_version = OpenAiVersion\n",
    "    openai.api_base = OpenAiBase\n",
    "\n",
    "    llm = AzureOpenAI(deployment_name=OpenAiDavinci,\n",
    "            temperature=temperature,\n",
    "            openai_api_key=OpenAiKey,\n",
    "            max_tokens=tokenLength,\n",
    "            batch_size=10, \n",
    "            max_retries=12)\n",
    "\n",
    "    logging.info(\"LLM Setup done\")\n",
    "    embeddings = OpenAIEmbeddings(deployment=OpenAiEmbedding, chunk_size=1, openai_api_key=OpenAiKey)\n",
    "elif embeddingModelType == \"openai\":\n",
    "    openai.api_type = \"open_ai\"\n",
    "    openai.api_base = \"https://api.openai.com/v1\"\n",
    "    openai.api_version = '2020-11-07' \n",
    "    openai.api_key = OpenAiApiKey\n",
    "    llm = OpenAI(temperature=temperature,\n",
    "            openai_api_key=OpenAiApiKey,\n",
    "            max_tokens=tokenLength)\n",
    "    embeddings = OpenAIEmbeddings(openai_api_key=OpenAiApiKey)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Because we have a large document, it is expected that we will run into Token limitation error with code below\n",
    "chainType = \"stuff\"\n",
    "summaryChain = load_summarize_chain(llm, chain_type=chainType)\n",
    "summary = summaryChain.run(docs)\n",
    "print(\"Summary: \", summary)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def callOpenAi(prompt):\n",
    "    response = openai.Completion.create(\n",
    "        engine=davincimodel,\n",
    "        prompt=prompt,\n",
    "        max_tokens=100,\n",
    "    )\n",
    "    return response['choices'][0]['text']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# for doc in docs:\n",
    "#     resp = callOpenAi(\"Generate summary for the following text : \" + doc.page_content)\n",
    "#     print(resp)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " Microsoft Fabric is a prerelease product that provides an all-in-one analytics solution for enterprises. It offers a comprehensive suite of services, such as data lake, data engineering, and data integration, and features such as Data Engineering, Data Factory, and Data Science experiences, and Power BI services. It also provides sorting and filtering capabilities, a Contact list feature, OneDrive feature, and sensitivity labels.\n"
     ]
    }
   ],
   "source": [
    "# Let's change now the chaintype from stuff to mapreduce and refine to see the summary\n",
    "chainType = \"map_reduce\"\n",
    "summaryChain = load_summarize_chain(llm, chain_type=chainType, return_intermediate_steps=True)\n",
    "summary = summaryChain({\"input_documents\": docs}, return_only_outputs=True)\n",
    "outputAnswer = summary['output_text']\n",
    "print(outputAnswer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "\n",
       "Microsoft Fabric is an all-in-one analytics solution for enterprises that covers everything from data movement to data science, Real-Time Analytics, and business intelligence. It offers a comprehensive suite of services, including data lake, data engineering, and data integration, all in one place. The platform is built on a foundation of Software as a Service (SaaS), which simplifies analytics needs. This page provides documentation on how to get started with Fabric, including tutorials, context-sensitive help, and how-to guides."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  Microsoft Fabric is a prerelease product that brings together components from Power BI, Azure Synapse, and Azure Data Explorer into a single integrated environment. It provides advantages such as an extensive range of deeply integrated analytics, shared experiences, easy access to assets, a unified data lake, and centralized administration and governance. IT teams can centrally configure core enterprise capabilities and data sensitivity labels are inherited automatically."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  Microsoft Fabric is a comprehensive set of analytics experiences tailored to specific personas and tasks. It includes Data Engineering, Data Factory, and Data Science experiences, which enable users to perform large scale data transformation, connect to data sources, and build, deploy, and operationalize machine learning models."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  Microsoft Fabric provides data science, data warehousing, real-time analytics, and Power BI services to empower data scientists and business analysts to move from descriptive to predictive insights. It offers industry-leading SQL performance and scale, natively stores data in the open Delta Lake format, and provides a best-in-class engine for observational data analytics. Power BI is the world's leading Business Intelligence platform to access data quickly and intuitively."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  Microsoft Fabric is an implementation of data mesh architecture that enables organizations and individuals to turn large and complex data repositories into actionable workloads and analytics. It unifies the OneLake and lakehouse architecture across enterprises, and is built on top of ADLS (Azure Data Lake Storage) Gen2. OneLake simplifies the experiences, eliminating the need for users to understand any infrastructure concepts, and eliminates data silos."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  OneLake is a unified storage system for developers that eliminates the need for provisioning and configuring isolated storage accounts. It is hierarchical in nature and provides a single-pane-of-glass file-system namespace that spans across users, regions and clouds. It is built into Microsoft Fabric and is divided into manageable containers for easy handling. Each tenant can create any number of workspaces, and each workspace can contain multiple lakehouses, which are collections of files, folders, and tables."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  Microsoft Fabric (Preview) trial offers access to the Fabric product experiences, including OneLake workspaces which allow users to ingest data into their own lakehouses, start processing, analyzing, and collaborating on the data, and access data stored in Azure Data Lake Storage using shortcuts. Additionally, shortcuts allow users to easily share data between users and applications without moving or duplicating information. The trial also includes access to end-to-end tutorials and Microsoft Fabric terminology."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  Microsoft Fabric is currently in preview and requires a Power BI license for the trial. Existing Power BI users can skip to starting the Fabric (Preview) trial, while new users must sign up for a Power BI free license first. The trial lasts until Fabric General Availability (GA) and is extended for 60 days after GA. To start the trial, users must open the Fabric homepage, select the Account manager, agree to the terms, and select Start trial. Once the trial capacity is ready, users receive a confirmation message and can begin working in Fabric."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  Microsoft Fabric (Preview) trial allows users to access all Fabric experiences and features, as well as OneLake storage up to 1 TB. It also allows users to create workspaces and collaborate with others in the same trial capacity. The size of the capacity is determined by the SKU and users can track the number of days remaining in their trial through their Account Manager."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  Fabric (Preview) trial allows users to share datasets, warehouses, and notebooks with other Fabric users and collaborate on them. It also allows users to create analytics solutions using these Fabric items. The trial provides one 64 capacity unit (CU) trial capacity which allows users to consume 64x60 CU seconds every minute. There is no limit on the number of workspaces or items created within the capacity, but the availability of capacity units and the rate of consumption are the only constraints."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  As the capacity owner for a Fabric (Preview) trial, you have access to a Capacity Metrics app for a detailed and transparent report on capacity consumption. Cancelling the trial will delete the capacity, workspaces, and contents, and will not allow users to create workspaces, share Fabric items, or create analytics solutions. Power BI administrators can enable and disable trials for paid features, and each trial user is the capacity admin for their trial capacity."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  Power BI administrators may have disabled access to the Fabric (Preview) trial, and existing Power BI trial users cannot start a trial from their Account manager. If the tenant has exhausted its limit of trial capacities, users can purchase a Fabric capacity from Azure, request another trial capacity user to share their trial capacity, or reach out to their Power BI administrator to increase tenant trial capacity limits. In Workplace settings, users cannot assign a workspace to the trial capacity."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  This known bug occurs when the Power BI administrator turns off trials after they have already been started. To add a workspace to the trial capacity, the Admin portal must be opened and the trial capacity must be chosen. The region of the Fabric (Preview) trial capacity is located in the home region of the tenant. When Fabric is Generally Available (GA), the Account manager can be used to start a trial and assign the Fabric capacity to a specific region. The Fabric (Preview) trial is different from an individual trial of Power BI paid as it allows access to the Fabric landing page."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  The Fabric (Preview) trial is a free trial for users that allows them to access non-Power BI experiences and items, with the caveat that they cannot create Fabric items if private links are enabled and public access is disabled. Autoscale is not supported in the trial capacity, but users can purchase a Fabric capacity in Azure if they need more compute capacity. Existing Synapse and Power BI users can sign up for the trial and start running product experiences immediately. An Azure subscription is not required to start the trial, but can be used to purchase a paid Fabric capacity."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  This article explains how to migrate existing workspaces into a trial capacity using workspace settings and choosing \"Trial\" as the license mode. It also provides definitions of terms used in Microsoft Fabric, including capacity, experience, and item."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  Synapse Data Engineering is a collection of items that bring together different functionality in a single environment designed for collaboration. It includes the lakehouse, notebook, and Spark job definition items. A lakehouse is a collection of files, folders, and tables that represent a database over a data lake used by the Apache Spark engine and SQL engine for big data processing. A notebook is a multi-language interactive programming tool with rich functions."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  Apache Spark is a program that helps data engineers and data scientists explore and process data, build machine learning experiments, and collaborate with the team. It can be easily transformed to a pipeline activity for orchestration. It consists of multiple jobs that run in parallel and is optimized with a write optimization to the parquet file format. Data Factory offers a rich set of connectors to connect."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  Data Factory is a platform for orchestrating data movement and transformation, with Dataflows providing a low-code interface for ingesting data from hundreds of sources and transforming it. Data Wrangler is a notebook-based tool for exploratory data analysis, and Experiments are the primary unit of organization and control for machine learning runs. Connectors are available for different types of data stores."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "\n",
       "MLflow is a machine learning model that is trained to recognize patterns in data sets. A run corresponds to a single execution of the model code. Synapse Data Warehouse is a traditional data warehouse with full transactional T-SQL capabilities. KQL database and KQL Queryset are used to query and manipulate data from the Data Explorer database."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  Microsoft Fabric is a platform that provides a centralized place to capture, transform, and route real-time events with a no-code experience. OneLake shortcuts provide a way to connect to existing data without having to directly copy it. End-to-end tutorials are available to guide users through the entire process from data acquisition to data consumption. Microsoft Fabric is currently in PREVIEW."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  This article provides tutorials for multiple Fabric experiences, such as Lakehouse, Data Science, Real-Time Analytics, and Data Warehouse. Each tutorial provides a scenario and instructions for how to build a dataflow, pipeline, dimensional model, and report. Microsoft does not provide any warranties with respect to the information provided."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  This article provides a reference guide and example scenarios to help decide whether to use a copy activity, dataflow, or Spark for workloads using Microsoft Fabric. It covers the primary developer persona, skill set, and code written for each use case. Microsoft Fabric is currently in PREVIEW."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "Dataflow Gen 2 is a platform for data ingestion, transformation, wrangling, and profiling. It supports data engineers, data integrators, and business analysts, and can be used with ETL, M, and SQL. It offers both code and low code options, with Spark as the main source and 30+ connectors, 150+ connectors, and hundreds of Spark libraries as destinations. It also supports low to high complexity transformations, with 300+ transformations and support for native Spark and open source libraries."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  Leo is a data engineer who is new to Spark and wants to process data on a schedule. He uses pipeline copy activity to move raw binary data from various sources into a consolidated lakehouse. Mary is a data engineer tasked with cleaning the data, applying business logics, and loading it into multiple destinations for reporting teams."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  Mary is an experienced Power Query user and decides to use Dataflow Gen 2 for her data transformation needs. Adam is a data engineer and uses Spark to build extract and transformation logic for customer review analytics. He writes a Spark application to read data from OneLake, cleanse and transform it, and write it to Delta tables in the lakehouse for downstream analytics."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  This article provides a guide to help users decide between using a data warehouse or lakehouse for their workloads using Microsoft Fabric. It compares the properties of each, such as data volume, type of data, primary developer persona, primary developer skill set, citizen developer, read and write operations, security, and primary development interface. It also provides an overview of the tools used for each, such as SQL, Spark, T-SQL, Dataflows, and Power BI Datamart."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  Susan, a professional developer, is new to Microsoft Fabric and needs to decide between building a data warehouse or a lakehouse. After reviewing the details, the primary decision points are the available skill set and the need for multi-table transactions. Susan has experience building data warehouses on relational database engines and is familiar with SQL syntax and functionality. The primary consumers of the data are also skilled with SQL and SQL analytical tools, so Susan decides to use a data warehouse which allows the team to interact primarily with T-SQL while also allowing any Spark users to access the data."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  Rob, a data engineer, needs to store and model several terabytes of data in Fabric. He decides to use a lakehouse, which allows the data engineering team to use their diverse skills against the data. Ash, a citizen developer, is a Power BI developer and needs to build a data product for a business unit. They decide to use a Premium capacity subscription, as the primary consumers of this data may be analysts familiar with no-code capabilities."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "\n",
       "This article provides an overview of navigating to items and actions from Microsoft Fabric Home. It explains the similarities that all product experiences share and how to create a warehouse and lakehouse in Microsoft Fabric. It also introduces Power BI datamarts and provides important information about the prerelease product."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  Microsoft Fabric is a platform that allows users to create, view, and share items such as apps, lakehouses, warehouses, and reports. Home is the best place to start working in Microsoft Fabric, but users can also navigate directly to a workspace using the nav pane and workspace selector. Home is personalized for each user and includes the left navigation pane, selector for switching product experiences, and global search. Power BI Home is different from the other product experiences."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  Microsoft Fabric offers many ways of locating and viewing content, such as searching or using the navigation pane. The top menu bar is for orienting oneself, finding items, help, and sending feedback. The Account manager control is important for looking up account information and managing the trial. Options for creating new items, links to recommended content, and items organized by recent, favorites, and items shared with you by colleagues are also available. Access to content is based on permissions, and if a subscription or license changes to one with less access, a prompt will appear asking to start a trial or upgrade the license."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  The Data Factory Nav Pane organizes actions you can take with your items and allows you to quickly find and open workspaces. Workspaces are places to collaborate with colleagues to create collections of items such as lakehouses, warehouses, and reports. You can search for a workspace by name or owner, or select the Workspaces icon in the nav pane and choose which workspace to open. The workspace name is listed on the canvas when it is opened."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  Microsoft Fabric provides a nav pane which includes notebooks, pipelines, reports, and lakehouses. It also includes an experience selector which allows users to open and make active other product experiences. Users can search, sort, and filter their content. The Help pane provides context sensitive help and suggested topics to learn how to use Microsoft Fabric features and terminology."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  The Help pane is a great place to search for answers to questions. It also provides links to ask the community for help, connect with Microsoft Fabric Support, and access the Account manager for account and license information. Notifications, settings, and feedback options can be found in the upper right corner of Home, as well as the canvas in the center of Home."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  Power BI Home is a feature that allows users to create new items, view recommended items, recents, favorites, and content shared with them. It also includes a Help pane which is feature-aware and provides answers to questions in the Fabric documentation and Fabric community forums. Microsoft Fabric is currently in PREVIEW."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  The Help pane is a search engine in Fabric that provides feature-aware documents, forum topics, and other resources. It is perfect for learning and getting started as it updates based on selections in Fabric. It allows users to find relevant information and resources from Microsoft documentation and community forum topics."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "\n",
       "Microsoft Fabric is currently in PREVIEW and the Help pane provides articles and instructions to help users learn about the data hub. Users can browse recent documents, search the Microsoft documentation and community forums, and submit questions or ideas for new features. Additionally, users can use search, filter, and sort functions to navigate long lists of content."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  Microsoft Fabric provides a search feature that allows users to quickly find items by title, name, or keyword. The search is available from Home and other areas of Microsoft Fabric, and users can also use the Filter by keyword field to narrow down the content on their canvas. Microsoft makes no warranties with respect to the information provided."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  Microsoft Fabric offers sorting and filtering capabilities to help users quickly locate content. Sorting is available for some columns, while filtering is available for content type, time, and owner. Fabric is currently in preview and the information provided may be subject to change."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  This article explains how to open the Fabric settings pane and the types of settings that can be accessed from there. Preferences can be set for individual users, such as language and account management. Item settings, developer settings, and Power BI settings can also be accessed. Additionally, users can manage personal/group storage from the resources and extensions section."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  Workspaces are collaborative places to create collections of Power BI items such as dashboards, datasets, workbooks, reports, datamarts, and dataflows. It also provides links to manage connections, on-premises data gateways, embed codes, virtual networks, Azure Analysis Services, migrations, governance and insights settings, the Fabric admin portal, and the Microsoft Purview hub."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  This article provides tips on how to manage workspaces, including pinning workspaces, setting workspace roles, navigating to the current workspace, and updating workspace settings. It also explains how to set up a workspace, and how to use the contact list and other settings to manage the workspace."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  The Contact list feature in Workspace settings allows users to specify which users receive notifications about issues in the workspace. The Workspace OneDrive feature allows users to configure a Microsoft 365 Group whose SharePoint document library is available to workspace users. Creating Microsoft 365 Groups may be restricted in certain environments and permissions between workspace access and Microsoft 365 Group membership are not synchronized."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  You can configure OneDrive in workspace settings by typing in the name of the Microsoft 365 group created earlier. Workspaces can be assigned to any capacity in your organization, and workspace admins can configure dataflow storage to use Azure Data Lake Gen 2 storage and Azure Log Analytics (LA) connection to collect usage and performance logs for the workspace. System storage is the place to manage your dataset storage in your individual or organization's capacity."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  This article provides information about how to manage a workspace account, including how to free up storage, delete the workspace, and audit activities. Admins can manage users' ability to create workspaces and manage the workspaces themselves in the Microsoft Fabric admin portal. Auditing is also available for activities such as creating and deleting folders."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  This article explains how to create workspaces in Microsoft Fabric, including how to give the workspace a unique name, provide a description, and assign it to a domain. It also outlines considerations and limitations, such as the maximum number of datasets and reports per workspace, and the use of URL encoding of special characters. Finally, it provides instructions on how to create and give access to workspaces."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  Advanced settings in Power BI allow users to add contacts to the contact list, change the license mode, and set the default storage format. Depending on the license mode, certain features may be enabled or disabled, and when downgrading from Premium capacity to Pro, any non-Power BI Fabric items must be removed. Large datasets can be enabled with the Large dataset storage format setting."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  Power BI template apps are developed for sharing outside an organization and data used with Power BI is stored in internal storage provided by Power BI. With the integration of dataflows and Azure Data Lake Storage Gen 2, users can store their dataflows in their organization's Azure Data Lake Storage Gen2 account. Workspace roles allow users to manage who can do what in a Microsoft Fabric workspace, and workspaces can be pinned to the top of the workspace flyout list for quick access."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  Microsoft Fabric workspaces sit on top of OneLake and divide the data lake into separate containers that can be secured independently. It extends the Power BI workspace roles by associating new Microsoft Fabric capabilities such as data integration and data exploration with existing workspace roles. Users in workspace roles have additional Microsoft Fabric capabilities in addition to the existing Power BI capabilities associated with these roles."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  This article discusses the capabilities of the KQL databases, KQL query-sets, and real-time dashboards, as well as how to connect to SQL endpoints of Lakehouse and Data warehouse. It also outlines the roles and permissions of admins, members, contributors, and viewers, and the steps needed to give users access to workspaces, such as creating workspaces, granting users access, and understanding OneLake security, OneLake shortcuts, Data warehouse security, Data engineering security, and Data science roles and permissions."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  Microsoft Fabric users can give others access to their workspace by assigning them different roles. Workspace creators are automatically admins and to enforce row-level security, viewers should be assigned the Viewer Role. To add or remove access for a user or group, the permission change only takes effect the next time the user logs into Microsoft Fabric. Access can be managed by selecting the Manage Access option on the command bar or More options menu."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  The OneLake data hub provides a filterable list of all the data items you can access, a gallery of recommended data items, a way of finding data items by workspace, and a way to display only the data items of a selected domain. This article explains how to use the data hub, including filtering the list of data items and using the options menu. It also describes the three tabs of the data items list: All, My Data, and Endorsed in your organization."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  The Endorsement overview provides a list of data items with columns that include Name, Endorsement status, Owner, Workspace, Last Refresh time, Next Refresh time, and Sensitivity. The Explorer pane can be used to view data items by workspace, and the tiles across the top of the data hub can be used to find and explore recommended data."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "\n",
       "The data hub allows users to view recommended data items, filter by domain, and open an item's options menu. Hovering over an item in the data items list will reveal the More options menu. The Explorer pane may list workspaces that the user does not have access to, but will still show items that the user has access to."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  Fabric provides two ways to endorse items: promotion and certification. Promotion is done by anyone with write permissions on the item, while certification is done by authorized reviewers. This article explains how to promote and certify items, and how to request certification if you are not an authorized reviewer."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  This article outlines the steps to promote and certify a Power BI dataset. It explains how to get write permissions on the item, review it, and select the Promoted or Certified option in the endorsement section. It also explains how to make the dataset discoverable and how to request item certification if the user is not authorized to certify it."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  This article provides instructions on how to apply sensitivity labels to Microsoft Fabric items. It explains the prerequisites and how to open the settings of an item, expand the endorsement section, and select the link about how to get the item certified. It also notes that if the link redirects back to the note, the Power BI admin has not made any information available and the user should contact them directly."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  To apply sensitivity labels to Fabric items, users must have Power BI Pro or Premium Per User (PPU) license, edit permissions on the item, and select the sensitivity indication in the header or open the item's settings to find the sensitivity section and choose the desired label. If the sensitivity label is greyed out, contact tech support."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# For the chaintype of MapReduce and Refine, we can also get insight into intermediate steps of the pipeline.\n",
    "# This way you can inspect the results from map_reduce chain type, each top similar chunk summary\n",
    "intermediateSteps = summary['intermediate_steps']\n",
    "for step in intermediateSteps:\n",
    "        display(HTML(\"<b>Chunk Summary:</b> \" + step))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " Microsoft Fabric is a prerelease product that provides a comprehensive suite of analytics experiences, including data engineering, data factory, data science, data warehouse, real-time analytics, and Power BI. It is built on a foundation of Software as a Service (SaaS) and offers features such as data promotion and certification, sensitivity labels, and workspace access control. Microsoft Fabric is currently in PREVIEW and provides a Home page as a central hub for accessing and viewing items such as apps, lakehouses, warehouses, reports, and more. It also offers a feature-aware search engine with recommended topics, resources, and links relevant to the user's current context and location. Applying sensitivity labels to items in Microsoft Fabric is a powerful way to protect sensitive content from unauthorized access and leakage, and requires a Power BI Pro or Premium Per User (PPU) license and edit permissions on the item.\n"
     ]
    }
   ],
   "source": [
    "from langchain.prompts import PromptTemplate\n",
    "# While we are using the standard prompt by langchain, you can modify the prompt to suit your needs\n",
    "promptTemplate = \"\"\"You are an AI assistant tasked with summarizing documents. \n",
    "        Your summary should accurately capture the key information in the document while avoiding the omission of any domain-specific words. \n",
    "        Please generate a concise and comprehensive summary that includes details. \n",
    "        Ensure that the summary is easy to understand and provides an accurate representation. \n",
    "        Begin the summary with a brief introduction, followed by the main points. \n",
    "        Please remember to use clear language and maintain the integrity of the original information without missing any important details:\n",
    "        {text}\n",
    "\n",
    "        \"\"\"\n",
    "customPrompt = PromptTemplate(template=promptTemplate, input_variables=[\"text\"])\n",
    "chainType = \"map_reduce\"\n",
    "summaryChain = load_summarize_chain(llm, chain_type=chainType, return_intermediate_steps=True, \n",
    "                                    map_prompt=customPrompt, combine_prompt=customPrompt)\n",
    "summary = summaryChain({\"input_documents\": docs}, return_only_outputs=True)\n",
    "outputAnswer = summary['output_text']\n",
    "print(outputAnswer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  Microsoft Fabric is an all-in-one analytics solution designed to simplify the data and analytics needs of enterprises. It offers a comprehensive suite of services, including data lake, data engineering, and data integration, all in one place, and is built on a foundation of Software as a Service (SaaS). With Fabric, users can enjoy a highly integrated, end-to-end, and easy-to-use product. It also provides features such as data promotion and certification, sensitivity labels, and workspace access control."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "Microsoft Fabric is a prerelease product that brings together components from Power BI, Azure Synapse, and Azure Data Explorer into a single integrated environment. It provides users with a range of experiences such as Data Engineering, Data Factory, Data Science, Data Warehouse, Real-Time Analytics, and Power BI on a shared SaaS foundation. This integration offers advantages such as an extensive range of deeply integrated analytics, shared experiences across experiences, easy access and reuse of assets, a unified data lake, and centralized administration and governance. IT teams can centrally configure core enterprise capabilities and data sensitivity labels are inherited automatically across the items in the suite."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "Microsoft Fabric is a comprehensive set of analytics experiences designed to work together seamlessly. It offers industry-leading experiences in data engineering, data factory, and data science. Data Engineering experience provides a world class Spark platform with great authoring experiences, enabling data engineers to perform large scale data transformation and democratize data through the lakehouse. Data Factory combines the simplicity of Power Query with the scale and power of Azure Data Factory, and more than 200 native connectors to connect to data sources on-premises and in the cloud. Data Science experience enables users to build, deploy, and operationalize machine learning models seamlessly within the Fabric experience, which integrates with Azure Machine Learning to provide built-in experiment tracking."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "Microsoft Fabric is a cloud-based platform that provides a comprehensive suite of data science, data warehousing, real-time analytics, and Power BI tools. It enables organizations to quickly and easily access and analyze data, allowing them to make better decisions. Data science capabilities include predictive analytics, machine learning, and model registry. Data Warehouse experience provides industry leading SQL performance and scale, with data stored in the open Delta Lake format. Real-Time Analytics is best in class engine for observational data analytics, which is collected from various sources such as apps, IoT devices, and human interactions. Finally, Power BI is the world's leading Business Intelligence platform, allowing business owners to access all the data in Fabric quickly and intuitively."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  Microsoft Fabric is a unified platform that offers a comprehensive big data analytics solution. It is an implementation of data mesh architecture, which unifies OneLake and lakehouse architecture across enterprises. OneLake is built on top of ADLS Gen2 and provides a single SaaS experience and a tenant-wide store for data that serves both professional and citizen developers. It eliminates data silos and does not require users to understand any infrastructure concepts or have an Azure account. Additionally, it simplifies experiences and provides a unified location to store all organizational data where the experiences operate."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "OneLake is a unified storage system for developers that simplifies discovery and data sharing while enforcing policy and security settings centrally and uniformly. It is hierarchical in nature and built into Microsoft Fabric, with no up-front provisioning required. OneLake provides a single-pane-of-glass file-system namespace that spans across users, regions, and clouds. Data is divided into manageable containers for easy handling, with each tenant mapping to the root of OneLake. Within a tenant, workspaces can be thought of as folders, and lakehouses are collections of files, folders, and tables that represent a database over a data lake."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "Microsoft Fabric is a cloud-based platform that provides users with a unified experience for data engineering, data warehousing, data factory, Power BI, and real-time analytics. It allows users to ingest data into their own lakehouses, start processing, analyzing, and collaborating on the data, just like OneDrive in Office. The experiences are prewired to OneLake, Microsoft's native store, and users can mount existing PaaS storage accounts into OneLake with the Shortcut feature. Shortcuts also allow users to share data between users and applications without moving or duplicating information, and extend to other storage systems, allowing users to compose and analyze data across clouds with transparent, intelligent caching. The Microsoft Fabric (Preview) trial includes access to the Fabric product experiences, as well as end-to-end tutorials and the ability to create a workspace and navigate to items from the Microsoft Fabric Home page."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "Microsoft Fabric is a prerelease product that requires a Power BI license to access the Fabric (Preview) trial. After signing up for a Power BI license, users can start the Fabric (Preview) trial by navigating to the Fabric homepage, selecting the Account manager, and selecting Start trial. After agreeing to the terms, users will receive a confirmation message when their trial capacity is ready. The trial lasts until Fabric General Availability (GA) and is extended for 60 days after GA."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "This summary provides an overview of the Microsoft Fabric (Preview) trial. It explains how to start a trial, the features included, and the compute resources available. It also outlines the benefits of the trial, such as full access to Fabric experiences and features, as well as OneLake storage up to 1 TB. Finally, it explains how to create workspaces for projects that support Fabric capabilities."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "Fabric (Preview) is a trial platform that allows users to share datasets, warehouses, and notebooks, and collaborate on them with other Fabric users. It also allows users to create analytics solutions using these Fabric items. To begin using the trial, users must add items to their workspace and assign it to the trial capacity using the \"Trial\" license mode. This trial capacity is provisioned with 64 capacity units (CUs), which allow users to consume 64x60 CU seconds every minute. The Fabric platform aggregates consumption from all experiences and applies it to the reserved capacity, with different functions having varying consumption rates. There is no limit on the number of workspaces or items that can be created, only the availability of capacity units and the rate at which they are consumed."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "This document outlines the process of administering a Fabric (Preview) trial. It explains how to cancel a trial, disable user access, and manage capacity units. Canceling a trial will delete the capacity, workspaces, and all associated items. Power BI administrators can enable and disable trials for paid features for Power BI and Fabric. Each trial user is the capacity admin for their trial capacity and Microsoft currently does not support multiple capacity administrators per trial capacity. Capacity admins have access to a detailed and transparent report for how capacity units are consumed via the Capacity Metrics app. Administering a Fabric trial requires careful evaluation of the impact of making changes to settings."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "Power BI administrators are unable to view metrics for individual capacities. However, an upcoming admin monitoring feature will enable administrators to view metrics for individual capacities. In addition, users are unable to start a trial if their Power BI administrator has disabled access, or if their tenant has exhausted its limit of trial capacities. Possible solutions include purchasing a Fabric capacity from Azure, requesting another trial capacity user to share their trial capacity with them, or reaching out to their Power BI administrator to create a CSS request to increase tenant trial capacity limits. Lastly, users are unable to assign a workspace to the trial capacity in Workplace settings."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "This document outlines the steps to take when setting up a Fabric (Preview) trial capacity. To add your workspace to the trial capacity, open the Admin portal by selecting it from the gear icon in the top menu bar. Then, select Trial > Capacity settings and choose the name of the capacity. The region of the trial capacity is determined by the home region of the tenant. If the home region doesn't have Fabric enabled, the trial won't be able to create any Fabric items. When Fabric is Generally Available, users can use the Account manager to start a trial and assign their Fabric capacity to a specific region. A per-user trial of Power BI paid allows access to the Fabric landing page."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "The Fabric (Preview) trial allows users to access non-Power BI experiences and items. All rules governing Power BI licenses and what can be done in the Power BI experience remain the same. During the Fabric preview, users cannot create Fabric items in the trial capacity if private links are enabled and public access is disabled. Autoscale is not supported in the Fabric (Preview) trial, but users can purchase a Fabric capacity in Azure if more compute capacity is needed. Existing Synapse users can sign up for a free trial and start running product experiences immediately, while existing Power BI users can access non-Power BI experiences and items. An Azure subscription is not required to start a Fabric (Preview) trial, but users can purchase a paid Fabric capacity if they have an existing Azure subscription."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "This article provides an overview of Microsoft Fabric terminology, including terms related to Synapse Data Warehouse, Synapse Data Engineering, Synapse Data Science, Synapse Real-Time Analytics, Data Factory, and Power BI. Fabric offers capacity through the Fabric SKU and Trials. Capacity is a dedicated set of resources available at a given time, and an experience is a collection of capabilities targeted to a specific functionality. An item is a set of capabilities within an experience, and users can migrate their existing workspaces into a trial capacity using workspace settings and choosing \"Trial\" as the license mode."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "This document provides an overview of the capabilities of Microsoft Synapse Data Engineering. It explains the concept of a tenant, workspace, lakehouse, and notebook. A tenant is a single instance of Fabric for an organization and is aligned with an Azure Active Directory. A workspace is a collection of items that brings together different functionality in a single environment designed for collaboration. Lakehouse is a collection of files, folders, and tables that represent a database over a data lake used by the Apache Spark engine and SQL engine for big data processing. Notebook is a multi-language interactive programming tool with rich functions such as authoring code and markdown, running and debugging code, and visualizing data."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  Apache Spark is a powerful tool for data engineers and data scientists to explore and process data, build machine learning experiments with both code and low-code experience, and transform it into a pipeline activity for orchestration. A Spark application is a program written by a user using one of Spark's API languages (Scala, Python, Spark SQL, or Java) or Microsoft-added languages (.NET with C# or F#). When an application runs, it's divided into one or more Spark jobs that run in parallel to process the data faster. A Spark job definition is a set of parameters, set by the user, indicating how a Spark application should be run. Data Factory offers a rich set of connectors that allow you to connect to various data sources, such as Azure Storage, Azure SQL Database, and Azure Cosmos DB. Additionally, it uses V-order, a write optimization to the parquet file format, to enable fast reads and provide cost efficiency and better performance."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "Data Factory is a platform for orchestrating data movement and transformation. It includes pipelines, Dataflow Gen2, Data Wrangler, and machine learning experiments. Dataflow Gen2 provides a low-code interface for ingesting data from hundreds of data sources and transforming data. Data Wrangler is a notebook-based tool that provides an immersive experience for exploratory data analysis. Machine learning experiments are the primary unit of organization and control for all related machine learning runs."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "This document provides an overview of machine learning models, runs, Synapse data warehousing, SQL Endpoint, Synapse Data Warehouse, KQL database, and KQL Queryset. A machine learning model is a file trained to recognize certain patterns. Runs correspond to a single execution of model code and are tracked in MLflow. Synapse data warehousing has a SQL Endpoint that allows a user to query delta table data with TSQL over TDS. Synapse Data Warehouse supports the full transactional T-SQL capabilities of an enterprise data warehouse. The KQL database is the representation of a database holding data in a format to execute a KQL query against it. The KQL Queryset is used to run queries, view results, and manipulate query results on data from the Data Explorer database. It allows users to save queries for future use or export and share queries with others."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "Microsoft Fabric is a platform that provides a centralized place to capture, transform, and route real-time events to destinations with a no-code experience. It also offers shortcuts to existing data stores and end-to-end tutorials to help users develop a foundational understanding of the Fabric UI, the various experiences supported by Fabric and their integration points, and the professional and citizen developer experiences available. The tutorials guide users through a scenario that covers the entire process, from data acquisition to data consumption. Microsoft Fabric is currently in PREVIEW."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "This document provides tutorials for multiple Fabric experiences, including Lakehouse, Data Science, Real-Time Analytics, and Data Warehouse. In the Lakehouse tutorial, users ingest, transform, and load data from a fictional retail company, Wide World Importers, into the lakehouse and analyze sales data across various dimensions. In the Data Science tutorial, users explore, clean, and transform a taxicab trip dataset, and build a machine learning model to predict trip duration. In the Real-Time Analytics tutorial, users use the streaming and query capabilities of Real-Time Analytics to analyze the New York Yellow Taxi trip dataset. In the Data Warehouse tutorial, users build an end-to-end data warehouse for Wide World Importers. Finally, in the Power BI tutorial, users build a dataflow and pipeline to bring data into a lakehouse, create a dimensional model, and generate a report. Microsoft makes no warranties, expressed or implied, with respect to the information provided."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "This guide provides an overview of the different capabilities of Microsoft Fabric, which is currently in PREVIEW. It covers the properties of the Copy Activity, Dataflow, and Spark, and provides example scenarios for each. Copy Activity is suitable for data lake and data warehouse migration, data ingestion, and lightweight transformation. Dataflow is suitable for ETL, SQL, and JSON. Spark is suitable for no code and low code. It is important to note that Microsoft Fabric is a prerelease product and may be subject to substantial modification before its release."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "For this scenario, Leo should use Dataflow Gen2 to ingest data from external systems. Dataflow Gen2 provides 30+ connectors for databases, file systems, and APIs, as well as 150+ connectors for data sources. It also offers a low-code development interface with a wizard and canvas, as well as a notebook and Spark job definition for more complex transformations. Dataflow Gen2 supports a wide range of data volumes from low to high, and provides hundreds of Spark libraries and 300+ transformations for native Spark and open-source functions."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "Mary is tasked with cleaning and loading data from multiple sources into a lakehouse and other destinations. She needs to apply business logic to the data and use copy activity to move petabytes of data from various sources, either ad-hoc or via a schedule. Mary will use pipeline copy activity to move the raw data into the lakehouse from Azure data resources and third-party sources, such as Snowflake Web, REST, AWS S3, GCS, etc. She will also use copy activity to load the data into a data warehouse with no code if the need arises. The copy activity is the best low-code and no-code choice for Mary to move petabytes of data to lakehouses and warehouses from varieties of sources."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "Adam is a data engineer tasked with building and maintaining data pipelines for a large retail company. He uses Spark to read structured, semi-structured, and unstructured data from OneLake for customer reviews and feedback. The application cleanses, transforms, and writes data to Delta tables in the lakehouse, making it ready for downstream analytics. With Spark's distributed computing platform, Adam is able to process large amounts of data in parallel to gain insights into customer experiences and improve services."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "This article provides a guide to help you choose between a data warehouse or lakehouse for your workloads using Microsoft Fabric. Data warehouses are designed for structured data and provide unlimited data volume. They are best for data warehouse developers and SQL engineers, and use T-SQL for read and write operations. Lakehouses are designed for unstructured, semi-structured, and structured data and also provide unlimited data volume. They are best for data engineers and data scientists, and use Spark (Scala, PySpark, Spark SQL, R) for read and write operations. Multi-table transactions are supported in data warehouses, but not in lakehouses. Power BI Datamarts are supported in both, with up to 100 GB of data. Microsoft Fabric is currently in PREVIEW."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "Susan should use a data warehouse for her project as it is the best fit for her and her team's skill set. It allows them to interact primarily with T-SQL while also allowing Spark users to access the data. Additionally, it provides the ability to perform multi-table transactions."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "Rob has decided to use a lakehouse to store and model several terabytes of data in Fabric. This allows the data engineering team to use their diverse skills against the data, while allowing the team members who are highly skilled in T-SQL to consume the data. Ash, a citizen developer, is a Power BI developer and needs to build a data product for a business unit. They review the details and decide that a self-service, no-code capability with data volume under 100 GB is the best solution. They already have a Premium capacity subscription and the primary consumers of the data will be analysts familiar with no-code."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  Microsoft Fabric is a cloud-based platform that provides a variety of data warehousing and SQL analytical tools. Ash decides to use a Power BI datamart, which allows the team to quickly build the capability with a no-code experience. Queries can be executed via Power BI and T-SQL, while also allowing access to the data for any Spark users in the organization. This article provides an overview of navigating to items and actions from Microsoft Fabric Home, which is currently in preview. Home displays items from all the workspaces that the user has access to, and provides detailed information about the product experience."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "Microsoft Fabric provides users with a Home page that serves as a central hub for accessing and viewing items such as apps, lakehouses, warehouses, reports, and more. Home is not workspace-specific, and the items that are available to the user are viewable on Home. To access Home, users can select it from the top of the left navigation pane. Home contains a variety of items and resources, including the left navigation pane, which links to different views of the user's items, and the selector for switching product experiences. Additionally, users can use global search to quickly find what they need. Power BI Home is different from the other product experiences, and users can learn more by visiting the Power BI Home page."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "This summary provides an overview of the features and functions of Microsoft Fabric. It outlines the top menu bar for orienting oneself, the Account Manager control for looking up account information, options for creating new items, links to recommended content, and items organized by recent, favorites, and items shared with others. It also explains that only the content that a user can access appears on their Home, with the exception of a prompt if their subscription or license changes. Finally, it mentions the navigation pane as a way of locating and viewing content, as well as the option of searching."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  nav pane. The Data Factory nav pane allows users to quickly access and open workspaces. It includes the Workspaces selector and My workspace, which is the active workspace. When a workspace is opened, its name replaces My workspace in the nav pane. This allows users to quickly create and access items in the workspace, such as lakehouses, warehouses, and reports. The nav pane also includes Home, Browse, OneLake data hub, and Create. Users can find and open workspaces by searching for the name or owner, or by selecting the Workspaces icon in the nav pane. Once opened, the workspace appears on the canvas and its name is listed on the nav pane."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "Microsoft Fabric is a product experience that provides users with the ability to find and open other product experiences, search, sort, and filter content, and access context sensitive help. The experience selector in the bottom left corner allows users to open and make active the desired experience. Global searching is available by item, name, keyword, workspace, and more. The Help pane provides users with suggested topics to learn about features and terminology, as well as community posts related to the current view."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "The Help pane in Microsoft Fabric is a great resource for finding answers to questions. It allows users to search for answers and provides links to ask the community for help or to connect with Microsoft Fabric Support. The Account manager provides information about the user's account and license. Notifications, settings, and feedback options can be found in the upper right corner of Home. The canvas in the center of Home is also a helpful resource for finding what you need."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "This article explains how to use the Fabric Help pane, a feature-aware search engine that quickly finds answers to questions in the Fabric documentation and Fabric community forums. The Help pane displays articles about the actions and features available on the current Fabric screen. It is an easy way to get help and find answers to questions related to Fabric. Microsoft Fabric is currently in PREVIEW, and this article provides information about the product before it is released."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "The Help pane is a search engine and feature-aware state that provides recommended topics, resources, and links relevant to the user's current context and location in Fabric. It has three sections: Feature-aware documents, Forum topics, and Other resources. It is perfect for learning and getting started, as the feature-aware documents update based on the user's selections and location in Fabric. Additionally, users can enter a keyword to find relevant information and resources from Microsoft documentation and community forum topics."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "The Help pane in Microsoft Fabric allows users to quickly access documents, forum posts, and other resources to learn about the data hub. It can be opened from the upper-right corner of Fabric, and provides Recent documents, Forum posts, Microsoft documentation, and other resources. Users can search for keywords, and submit questions or ideas for new features. Additionally, the Support site can be accessed from Other Resources. Searching, filtering, and sorting are helpful when dealing with long lists of content. Microsoft Fabric is currently in PREVIEW."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "Microsoft Fabric provides a global search box at the top of Home to quickly find items by title, name, or keyword. Search is available from Home and most other areas of Microsoft Fabric, and users can enter all or part of the name, creator, keyword, or workspace to find matches in all items they own or have access to. Additionally, most experiences on the Microsoft Fabric canvas include a Filter by keyword field to narrow down content on the canvas."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "This article provides an overview of sorting and filtering content lists in Microsoft Fabric. It explains how to sort content lists alphabetically, by date, or by other criteria, and how to filter content lists by type, time, or owner. Microsoft Fabric is currently in PREVIEW and this article provides important information about the prerelease product."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "The Fabric settings pane provides users with access to various settings to configure their personal preferences, language, notifications, and item settings. In the preferences section, users can set their user preferences, specify the language of the Fabric user interface, manage their account and notifications, and configure settings for their personal use. In the resources and extensions section, users can access the personal/group storage management page and the Power BI settings page."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "This article provides an overview of workspaces in Power BI, which are places to collaborate with colleagues to create collections of Power BI items such as dashboards, datasets, workbooks, reports, datamarts, and dataflows. It outlines the features available to manage connections, on-premises data gateways, embed codes, and virtual networks data gateways, as well as the ability to migrate Azure Analysis Services datasets to Power BI Premium. It also provides links to the Fabric admin portal, Microsoft Purview hub, and documentation to help admins and users with their admin, governance, and compliance tasks."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "This article provides an overview of workspaces, including how to manage access, pin workspaces to the top of the workspace flyout list, use granular workspace roles for flexible permissions management, and navigate to the current workspace from anywhere. Additionally, workspace admins can use workspace settings to manage and update the workspace, including general settings, contact list, OneDrive, license, Azure connections, storage, and other experiences' specific settings."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "This document provides an overview of the workspace settings and contact list features in Microsoft Fabric. It also explains how to configure a Microsoft 365 Group whose SharePoint document library is available to workspace users. The document also notes that creating Microsoft 365 Groups may be restricted in some environments, and that permissions between workspace access and Microsoft 365 Group membership are not synchronized. It is recommended to give access to the workspace to the same Microsoft 365 Group whose file library is available."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "This document outlines how to configure OneDrive, license mode, Azure connections, and system storage for Microsoft 365 workspaces. To configure OneDrive, type in the name of the Microsoft 365 group that was created earlier. License mode can be configured while creating a workspace or in Workspace settings -> Premium. Azure connections can be configured to use Azure Data Lake Gen 2 storage and Azure Log Analytics (LA) connection to collect usage and performance logs for the workspace. System storage is used to manage dataset storage in individual or shared workspaces."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "This article provides an overview of managing and auditing workspaces in Microsoft Fabric. Workspace storage includes datasets, Excel reports, and items shared by others. As an admin, you can delete the workspace, which removes the associated app from AppSource and deletes everything contained within the workspace for all group members. Microsoft Fabric admins decide who can create workspaces and distribute apps. Admins can also manage, recover, and delete workspaces. Microsoft Fabric audits activities such as creating and deleting folders."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "This article explains how to create workspaces in Microsoft Fabric. Workspaces are collections of items such as lakehouses, warehouses, and reports. To create a workspace, select Workspaces > New workspace. The Create a workspace pane opens, where you can give the workspace a unique name (mandatory) and provide a description (optional). Additionally, domain contributors can associate the workspace to a domain (optional). When done, either continue to the advanced settings, or select Apply. Limitations to be aware of include a maximum of 1,000 datasets or 1,000 reports per dataset, and special characters not supported in workspace names when using an XMLA endpoint. A user or a service principal can be a member of up to 1,000 workspaces."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "This document provides an overview of the advanced settings available for Power BI workspaces. It explains the Contact List, which is a place where users can add contacts to receive system email notifications for workspace level changes. It also covers the License Mode, which provides different sets of features for the workspace, and the Default Storage Format, which enables fast user interactivity and large datasets beyond the default limit with Premium capacities. Finally, it notes that when downgrading from Premium capacity to Pro (Shared capacity), any non-Power BI Fabric items must be removed first."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "Power BI template apps are developed for sharing outside an organization. Template app workspaces are created when the option is checked, and it is not possible to revert back to a normal workspace. Data used with Power BI is stored in internal storage provided by Power BI, but with the integration of dataflows and Azure Data Lake Storage Gen 2 (ADLS Gen2), dataflows can be stored in an organization's Azure Data Lake Storage Gen2 account. To collaborate with other users, they must be given access to the workspace and assigned roles. Pinning workspaces to the top of the workspace flyout list allows for quick access to favorite workspaces. Roles in workspaces allow for the management of who can do what in a Microsoft Fabric workspace."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "Microsoft Fabric workspaces provide a secure way to divide data lakes into separate containers that can be accessed independently. Each workspace can be assigned roles to individuals or to security groups, Microsoft 365 groups, and distribution lists. These roles include Admin, Member, Contributor, and Viewer, each of which grants different levels of access to the workspace. In addition to the existing Power BI capabilities associated with these roles, users in workspace roles have the ability to view and read content of data pipelines, notebooks, Spark job definitions, ML models and experiments, and Event streams."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "This article provides an overview of roles and permissions in workspaces in Power BI. It explains how to create workspaces, give users access to workspaces, and how to manage OneLake security, OneLake shortcuts, Data warehouse security, Data engineering security, and Data science roles and permissions. Admins, Members, and Contributors can grant viewers granular SQL permissions to read Lakehouse and Data warehouse data through SQL endpoints, and additional permissions are needed to read data from shortcut destination. This article provides a comprehensive guide to understanding and managing roles and permissions in Power BI workspaces."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "This article explains how to give access to a Microsoft Fabric workspace. Workspace creators are automatically admins, and admins or members can give others access by assigning them to different roles. To enforce row-level security, assign users the Viewer Role. Access changes only take effect the next time the user logs in. To add people or groups, select Manage Access on the workspace page, enter name or email, select a role, and select Add. Access can be viewed and modified later if needed. Finally, read about the workspace experience and create workspaces."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "The OneLake data hub is a powerful tool for discovering data items within an organization. It provides a filterable list of all the data items that the user has access to, a gallery of recommended data items, a way of finding data items by workspace, and a way to display only the data items of a selected domain. Additionally, the options menu of the data hub allows users to do various tasks with the data item. To open the data hub, users can select the OneLake data hub icon in the navigation pane. The data items list has three tabs: All, My Data, and Endorsed in Your Organization. All displays all data items the user has access to, My Data displays data items the user owns, and Endorsed in Your Organization displays endorsed data items in the user's organization. Certified data items are listed first, followed by promoted data items."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "This overview provides information about the Endorsement column in the data items list. The list contains the Name, Endorsement status, Owner, Workspace, Refreshed, Next refresh, and Sensitivity of each data item. The Explorer pane can be used to view data items by workspace, and the tiles across the top of the data hub can be used to find and explore recommended data items. Access to certain workspaces may be restricted, but users will still be able to view the items they have access to."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "This document provides information about recommended data items, domains, and how to open an item's options menu in the data hub. Recommended data items are certified or promoted by someone in the organization, recently refreshed or accessed. Domains can be used to filter the data hub to show only items belonging to that domain. To open an item's options menu, select More options (...) on the item. Hovering over the item will reveal the option. Note that the Explorer pane may list workspaces that the user does not have access to if the workspace contains items that the user does have access to."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "Fabric provides two ways to endorse items to increase visibility: promotion and certification. Promotion is a way to highlight items that are valuable and worthwhile for others to use, and is available to anyone with write permissions on the item. Certification is a way to show that an item meets the organization's quality standards and is reliable, authoritative, and ready for use across the organization. Currently, all Fabric items except Power BI dashboards can be endorsed. Promotion is available to anyone with write permissions on the item, while certification is only available to authorized reviewers. This article outlines how to promote and certify items, as well as how to request certification if you are not an authorized reviewer."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b>  open the settings of the item.\n",
       "\n",
       "2. Expand the endorsement section and select Request certification.\n",
       "\n",
       "3. Select Apply.\n",
       "\n",
       "Summary: Item certification is an important responsibility that only authorized users can perform. However, other users can request certification by going to the workspace where the item is located and opening the settings of the item. They can then expand the endorsement section and select Request certification. For Power BI datasets, users can also make the dataset discoverable, allowing users without access to the dataset to find it. Finally, authorized users can certify the item by getting write permissions on the item, carefully reviewing it, and selecting Certified in the endorsement section."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "This article assumes that you have the following prerequisites:\n",
       "\n",
       "• A Power BI tenant with sensitivity labels configured.\n",
       "• A Microsoft Fabric item.\n",
       "• Access to the Power BI admin portal.\n",
       "\n",
       "Summary: Microsoft Fabric is a prerelease product that allows users to apply sensitivity labels to their items in order to protect sensitive content from unauthorized access and leakage. This article provides instructions on how to apply sensitivity labels to Fabric items, including prerequisites such as a Power BI tenant with sensitivity labels configured, a Microsoft Fabric item, and access to the Power BI admin portal."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<b>Chunk Summary:</b> \n",
       "This document outlines the requirements for applying sensitivity labels to items in Fabric. In order to apply a sensitivity label, users must have a Power BI Pro or Premium Per User (PPU) license and edit permissions on the item they wish to label. If a user cannot apply a sensitivity label, or if the sensitivity label is greyed out in the sensitivity label menu, they may not have the necessary permissions and should contact their organization's tech support. There are two ways to apply a sensitivity label: from the flyout menu in the item header, and in the item settings. After applying the label, users can learn more about sensitivity labels in the overview."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# For the chaintype of MapReduce and Refine, we can also get insight into intermediate steps of the pipeline.\n",
    "# This way you can inspect the results from map_reduce chain type, each top similar chunk summary\n",
    "intermediateSteps = summary['intermediate_steps']\n",
    "for step in intermediateSteps:\n",
    "        display(HTML(\"<b>Chunk Summary:</b> \" + step))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}