pranay27sy/maritime_documents_tags_classification

<h1>Data Understanding and Preparation</h1>

<h2>Data Collection</h2> <ul> <li><strong>BIMCO Contracts and Clauses</strong> <ul> <li><strong>URL:</strong> <a href="https://www.bimco.org/contracts-and-clauses/bimco-contracts">BIMCO Contracts and Clauses</a></li> <li><strong>Description:</strong> This website provides a wide range of standardized contracts and clauses commonly used in the maritime industry. The documents were downloaded and used as part of our dataset, offering detailed insights into industry-specific terminology and structured data.</li> </ul> </li> </ul>

<h2>Data Description</h2> <ul> <li><strong>Documents Collected:</strong> A total of 217 documents were collected for this project.</li> <li><strong>Source:</strong> All 217 documents were sourced from the <a href="https://www.bimco.org/contracts-and-clauses/bimco-contracts">BIMCO Contracts and Clauses</a> website.</li> <li><strong>Document Types:</strong> The documents collected are associated with multiple tags provided by BIMCO. Below are some examples: <ul> <li>SHIP FINANCING</li> <li>TIME CHARTER</li> <li>TANKER</li> <li>SECURITY</li> <li>BAREBOAT CHARTER</li> <li>VOYAGE CHARTER</li> <li>CHEMTANKVOYBILL</li> <li>CREWING</li> <li>SALE AND PURCHASE</li> <li>PASSENGER</li> </ul> </li> </ul>

<h2>Document Structure</h2> <p>The documents vary in structure and format, ranging from standard templates to more detailed, customized layouts. Each document type contains specific fields relevant to its purpose in the shipping or trade process.</p>

<h2>Data Characteristics</h2> <ul> <li><strong>Language:</strong> All documents are in English.</li> <li><strong>Format:</strong> The documents are in PDF format.</li> <li><strong>Content Variety:</strong> The content ranges from simple textual entries (e.g., dates, names, quantities) to more complex structured data involving tables, clauses, and legal terminologies.</li> </ul>

<h2>Data Cleaning</h2> <p>No data cleaning was performed on the dataset to preserve the original content. This decision was made to ensure that the fine-tuned model can accurately classify documents as they are presented in real-world scenarios, capturing all nuances, variations, and inconsistencies in the data. By retaining the raw data, the model will be better equipped to handle diverse document structures, spelling variations, and other inconsistencies that may appear in actual usage.</p>

<h2>Data Preprocessing</h2> <p>An instruction dataset is a curated collection of instructions paired with their corresponding outputs or actions. These datasets are crucial for training or fine-tuning models, enabling them to understand and perform specific tasks. In the context of language models, instruction datasets are utilized to enhance the models' ability to follow commands and generate relevant outputs.</p>

<h2>Dataset Creation</h2> <ul> <li><strong>Instruction:</strong> The instruction provided for the dataset was: <code>&lt;instruction&gt;Predict the relevant tags for the following text segment, which pertains to a document from the maritime industry.&lt;/instruction&gt;</code> </li> <li><strong>Input Specification:</strong> Each input consisted of a document chunk with a maximum size of 4000 characters.</li> <li><strong>Output Specification:</strong> The output for each document chunk consisted of tags.</li> </ul>

<h2>Samples</h2> <p>A total of 1,783 samples were created during this process.</p>