---
license: mit
datasets:
- abisee/cnn_dailymail
language:
- en
metrics:
- rouge
- bleu
base_model:
- google-t5/t5-small
pipeline_tag: summarization
library_name: transformers
---
# Model Card for t5_small Summarization Model

## Model Details

- Model Architecture: T5 (Text-to-Text Transfer Transformer)
- Variant: t5-small
- Task: Text Summarization
- Framework: Hugging Face Transformers

## Training Data

- Dataset: CNN/DailyMail
- Content: News articles and their summaries
- Size: Approximately 300,000 article-summary pairs

## Training Procedure

- Fine-tuning method: Using Hugging Face Transformers library
- Hyperparameters:
  - Learning rate: 5e-5
  - Batch size: 8
  - Number of epochs: 3
- Optimizer: AdamW

## How to Use

1. Install the Hugging Face Transformers library:
```
pip install transformers
```

2. Load the model:
```python
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("t5-small")
model = AutoModelForSeq2SeqLM.from_pretrained("t5-small")
```

3. Generate a summary:
```python
input_text = "Your input text here"
inputs = tokenizer("summarize: " + input_text, return_tensors="pt", max_length=512, truncation=True)
summary_ids = model.generate(inputs["input_ids"], max_length=150, min_length=40, length_penalty=2.0, num_beams=4, early_stopping=True)
summary = tokenizer.decode(summary_ids[0], skip_special_tokens=True)
```

## Evaluation

- Metric: ROUGE scores (Recall-Oriented Understudy for Gisting Evaluation)
- Exact scores not available, but typically evaluated on:
  - ROUGE-1 (unigram overlap)
  - ROUGE-2 (bigram overlap)
  - ROUGE-L (longest common subsequence)

## Limitations

- Performance may be lower compared to larger T5 variants
- Optimized for news article summarization, may not perform as well on other text types
- Limited to input sequences of 512 tokens
- Generated summaries may sometimes contain factual inaccuracies

## Ethical Considerations

- May inherit biases present in the CNN/DailyMail dataset
- Not suitable for summarizing sensitive or critical information without human review
- Users should be aware of potential biases and inaccuracies in generated summaries
- Should not be used as a sole source of information for decision-making processes