# CSV Data Import

Import structured tabular data from CSV files to provide your AI agents with searchable databases, product catalogs, pricing information, and other structured datasets.

## Overview

| Property     | Details                       |
| ------------ | ----------------------------- |
| **Type**     | Static                        |
| **Refresh**  | Manual                        |
| **Tier**     | 2 (Professional & Enterprise) |
| **Format**   | CSV with any column structure |
| **Encoding** | UTF-8 recommended             |

## When to Use Data CSV

The Data CSV connector is ideal for:

* **Product Catalogs** - Product names, descriptions, SKUs, pricing
* **Employee Directories** - Staff information and contact details
* **Inventory Data** - Stock levels, warehouse locations
* **Pricing Tables** - Service tiers, feature matrices
* **Location Data** - Store addresses, hours, contact information
* **Statistical Data** - Reports, metrics, KPIs

## Difference from QnA CSV

| Feature  | QnA CSV                     | Data CSV                   |
| -------- | --------------------------- | -------------------------- |
| Purpose  | Question-Answer pairs       | Structured tabular data    |
| Columns  | Must have Question & Answer | Any column structure       |
| Use Case | FAQs, knowledge base        | Databases, catalogs, lists |
| Tier     | 1 (All plans)               | 2 (Professional+)          |

## CSV File Format

### Flexible Column Structure

Unlike QnA CSV, Data CSV accepts any column structure:

```csv
ProductName,SKU,Price,Category,InStock
Widget Pro,WGT-001,99.99,Electronics,Yes
Gadget Plus,GDG-002,149.99,Electronics,No
Service Bundle,SVC-100,299.99,Services,Yes
```

### Multi-Column Examples

**Employee Directory:**

```csv
Name,Department,Email,Phone,Location
John Smith,Engineering,john.smith@example.com,555-0100,Building A
Jane Doe,Sales,jane.doe@example.com,555-0101,Building B
```

**Store Locations:**

```csv
StoreName,Address,City,State,Zip,Phone,Hours
Main Street Store,123 Main St,Springfield,IL,62701,555-1000,Mon-Fri 9AM-6PM
Downtown Branch,456 Oak Ave,Capital City,IL,62702,555-2000,Mon-Sat 8AM-8PM
```

**Pricing Table:**

```csv
Plan,MonthlyPrice,AnnualPrice,Users,Storage,Support
Starter,29,290,5,10GB,Email
Professional,99,990,25,100GB,Email+Chat
Enterprise,299,2990,Unlimited,Unlimited,24/7 Phone
```

## How to Add Data CSV

### Step 1: Navigate to Data Sources

1. Log in to your Twig AI account
2. Click **Data** in the main navigation menu
3. Click **Add Data Source** or the **+** button

### Step 2: Select Data CSV Connector

1. Choose **Data as a CSV** from the list
2. The connector shows: "Tabular data as a CSV file"
3. Note: Available on Professional and Enterprise plans

### Step 3: Configure the Data Source

#### Basic Information

* **Name** (required): Descriptive name for your dataset
  * Example: "Product Catalog 2024", "Employee Directory", "Store Locations"
* **Description** (optional): Context about the data
  * Example: "Complete product catalog with pricing and availability"

#### File Upload

1. Click **Choose File** or drag-and-drop your CSV file
2. Upload your `.csv` file
3. Wait for upload to complete

#### Tags (Optional)

* Add tags for organization
* Examples: "product", "pricing", "internal", "public"

### Step 4: Save and Process

1. Click **Save** or **Create**
2. System will parse and index your tabular data
3. Monitor the processing status

### Step 5: Verify Import

1. Check record count matches CSV rows
2. Verify status shows "END\_PROCESS"
3. Test by asking questions about the data

## Creating Effective Data CSVs

### 1. Use Clear Column Headers

**Good:**

```csv
ProductName,Description,Price,Availability
```

**Avoid:**

```csv
col1,col2,col3,col4
p_name,desc,$$,avail
```

### 2. Consistent Data Formatting

**Good:**

```csv
Product,Price,Date
Widget A,99.99,2024-01-15
Widget B,149.99,2024-01-16
```

**Avoid:**

```csv
Product,Price,Date
Widget A,$99.99,Jan 15 2024
Widget B,149.99 USD,2024/01/16
```

### 3. Include Descriptive Information

**Good:**

```csv
ProductName,Description,Features,Benefits
Premium Widget,"High-quality widget with advanced features","Durable, Fast, Efficient","Saves time, Reduces costs"
```

**Better:**

```csv
ProductName,FullDescription
Premium Widget,"The Premium Widget is a high-quality solution designed for professional use. Features include durable construction, fast processing, and efficient operation. Benefits: Saves time through automation, Reduces operational costs by 30%."
```

### 4. Handle Missing Data

Use consistent representations for missing data:

```csv
Product,Price,DiscountPrice,Notes
Widget A,99.99,79.99,On sale this week
Widget B,149.99,,Regular price
Widget C,199.99,N/A,No discount available
```

## Examples by Use Case

### Example 1: Product Catalog

```csv
ProductName,SKU,Category,Price,Description,InStock,ShippingWeight
Deluxe Widget,DLX-001,Widgets,299.99,"Premium widget with advanced features including X, Y, and Z",Yes,2.5 lbs
Standard Gadget,STD-002,Gadgets,149.99,"Reliable gadget for everyday use with features A and B",Yes,1.2 lbs
Pro Service Pack,PRO-100,Services,499.99,"Complete service package including setup, training, and 6-month support",Yes,N/A
```

**Questions AI can answer:**

* "What widgets do you have?"
* "How much is the Deluxe Widget?"
* "What's in stock?"
* "Show me products under $200"
* "What services do you offer?"

### Example 2: Office Locations

```csv
OfficeName,Address,City,State,Zip,Country,Phone,Email,Hours,Specialization
Headquarters,123 Tech Boulevard,San Francisco,CA,94105,USA,555-0100,sf@example.com,Mon-Fri 8AM-6PM,"Sales, Support, R&D"
East Coast Office,456 Innovation Drive,Boston,MA,02108,USA,555-0200,boston@example.com,Mon-Fri 9AM-5PM,"Sales, Support"
European Hub,789 Tech Street,London,N/A,EC1A 1BB,UK,+44-20-5555-0300,london@example.com,Mon-Fri 9AM-5PM GMT,"Sales, Support, EU Operations"
```

**Questions AI can answer:**

* "Where are your offices located?"
* "What's the phone number for the Boston office?"
* "What are the hours for the London office?"
* "Which offices provide R\&D services?"

### Example 3: Pricing Plans

```csv
PlanName,MonthlyPrice,AnnualPrice,MaxUsers,StorageGB,Features,Support,BestFor
Starter,29.99,299.90,5,10,"Basic features, Email integration, Mobile app",Email only,"Small teams, Basic needs"
Professional,99.99,999.90,25,100,"All Starter features plus Advanced analytics, API access, Integrations, Custom branding",Email + Chat,"Growing teams, Professional use"
Enterprise,Custom,Custom,Unlimited,Unlimited,"All Professional features plus SSO, Advanced security, Dedicated account manager, Custom integrations, SLA",24/7 Phone + Dedicated support,"Large organizations, Complex requirements"
```

**Questions AI can answer:**

* "What plans do you offer?"
* "What's included in the Professional plan?"
* "How much does the Starter plan cost annually?"
* "Which plan is best for a small team?"
* "Do you offer SSO?"

### Example 4: FAQ + Metadata

```csv
Question,Answer,Category,LastUpdated,RelatedProducts
How do I reset my password?,"Go to login page, click 'Forgot Password', and follow email instructions.",Account,2024-01-15,N/A
What's your return policy?,"30-day returns on unused items with original packaging.",Policies,2024-01-10,"All products"
How do I contact support?,"Email: support@example.com, Phone: 1-800-555-0100, Live Chat on website",Support,2024-01-20,N/A
```

## Best Practices

### 1. Data Organization

**Denormalize for AI:** Instead of:

```csv
ProductID,Name
1,Widget A
2,Widget B
```

and separate

```csv
ProductID,Price
1,99.99
2,149.99
```

Use:

```csv
ProductID,Name,Price,Description
1,Widget A,99.99,Full description...
2,Widget B,149.99,Full description...
```

### 2. Include Context in Columns

Add descriptive columns that provide context:

```csv
Product,Price,WhenToUse,WhoItsFor
Basic Plan,29.99,"For simple needs and getting started","Individuals and small teams up to 5 people"
Pro Plan,99.99,"When you need advanced features and more capacity","Growing businesses with 10-50 employees"
```

### 3. Use Full Text, Not Codes

**Good:**

```csv
Product,Status,Availability
Widget A,In Stock,Ships within 24 hours
Widget B,Low Stock,Limited quantities available
Widget C,Backordered,Ships in 2-3 weeks
```

**Avoid:**

```csv
Product,Status,Availability
Widget A,IS,24H
Widget B,LS,LQ
Widget C,BO,2-3W
```

### 4. Consistent Formatting

* **Dates:** Use ISO format (YYYY-MM-DD)
* **Currency:** Include symbols ($99.99 or 99.99 USD)
* **Boolean:** Use Yes/No, True/False, or 1/0 consistently
* **Lists:** Use semicolons or pipes to separate multiple values

```csv
Product,Price,Features,Available
Widget A,$99.99,"Feature 1; Feature 2; Feature 3",Yes
Widget B,$149.99,"Feature X; Feature Y; Feature Z",Yes
```

## Technical Requirements

### File Format

* **Extension:** `.csv`
* **Encoding:** UTF-8 (recommended)
* **Delimiter:** Comma (,)
* **Line Endings:** LF (\n) or CRLF (\r\n)

### Size Limits

* Varies by plan
* Split large datasets into multiple CSVs if needed
* Consider sampling for very large datasets

### Headers

* Must have header row with column names
* No duplicate column names
* Avoid special characters in column names

## Updating Your Data CSV

To update your data:

1. Edit your local CSV file with changes
2. Navigate to the data source in Twig
3. Click **Edit**
4. Upload the updated CSV file
5. Click **Save** to reprocess

**Note:** The entire dataset is replaced, not merged.

## Troubleshooting

### Data Not Searchable

**Problem:** AI can't find information in the CSV

**Solutions:**

* Verify column headers are descriptive
* Add more descriptive text in columns
* Include context columns explaining the data
* Check encoding is UTF-8

### Incorrect Data Interpretation

**Problem:** AI misinterprets the data

**Solutions:**

* Use full text instead of abbreviations
* Add descriptive column names
* Include units in data (e.g., "Price USD" instead of "Price")
* Add context columns

### Processing Errors

**Problem:** CSV fails to process

**Solutions:**

* Check for malformed CSV (missing quotes, unescaped commas)
* Verify file encoding
* Look for special characters
* Ensure consistent column count across rows

### Missing Rows

**Problem:** Some rows not imported

**Solutions:**

* Check for empty required fields
* Verify no hidden characters
* Ensure no duplicate headers
* Review process logs for specific errors

## Advanced Use Cases

### 1. Multi-Language Support

```csv
ProductName,DescriptionEN,DescriptionES,DescriptionFR,Price
Widget A,"High-quality widget","Widget de alta calidad","Widget de haute qualité",99.99
```

### 2. Rich Product Information

```csv
Product,Description,Specifications,UseCases,Compatibility,Warranty
Premium Widget,"Detailed description...","Size: 10x10x5, Weight: 2lbs, Material: Aluminum","Perfect for X, Y, and Z scenarios","Works with Product A, B, and C","2-year limited warranty"
```

### 3. Hierarchical Data (Flattened)

```csv
Category,Subcategory,Product,Description,Price
Electronics,Computers,Laptop A,"13-inch laptop with...",999.99
Electronics,Computers,Laptop B,"15-inch laptop with...",1299.99
Electronics,Tablets,Tablet X,"10-inch tablet with...",499.99
```

## Next Steps

After importing your Data CSV:

1. Test queries to verify AI can access the data
2. [Create specialized agents](/product/overview/add-an-ai-agent-persona.md) for specific data types
3. [Monitor usage analytics](/product/monitoring/view-analytics.md) to see how data is queried
4. Refine and expand your CSV based on user questions

## Related Connectors

* [QnA CSV](/product/data-integrations/qna-csv.md) - Question-answer pairs
* [Files](/product/data-integrations/files.md) - Upload document files
* [Confluence](/product/data-integrations/confluence.md) - Structured wiki content
* [Google Drive](/product/data-integrations/google-drive.md) - Sync spreadsheets from Drive


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://help.twig.so/product/data-integrations/data-csv.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.