Scribbler310's picture
feat: enhance dashboard
c2b7eb3 verified
# @langchain/google-genai
This package contains the LangChain.js integrations for Gemini through their generative-ai SDK.
## Installation
```bash npm2yarn
npm install @langchain/google-genai @langchain/core
```
This package, along with the main LangChain package, depends on [`@langchain/core`](https://npmjs.com/package/@langchain/core/).
If you are using this package with other LangChain packages, you should make sure that all of the packages depend on the same instance of @langchain/core.
You can do so by adding appropriate field to your project's `package.json` like this:
```json
{
"name": "your-project",
"version": "0.0.0",
"dependencies": {
"@langchain/core": "^0.3.0",
"@langchain/google-genai": "^0.0.0"
},
"resolutions": {
"@langchain/core": "^0.3.0"
},
"overrides": {
"@langchain/core": "^0.3.0"
},
"pnpm": {
"overrides": {
"@langchain/core": "^0.3.0"
}
}
}
```
The field you need depends on the package manager you're using, but we recommend adding a field for the common `yarn`, `npm`, and `pnpm` to maximize compatibility.
## Chat Models
This package contains the `ChatGoogleGenerativeAI` class, which is the recommended way to interface with the Google Gemini series of models.
To use, install the requirements, and configure your environment.
```bash
export GOOGLE_API_KEY=your-api-key
```
Then initialize
```typescript
import { ChatGoogleGenerativeAI } from "@langchain/google-genai";
import { HumanMessage } from "@langchain/core/messages";
const model = new ChatGoogleGenerativeAI({
model: "gemini-pro",
maxOutputTokens: 2048,
});
const response = await model.invoke(new HumanMessage("Hello world!"));
```
#### Multimodal inputs
Gemini vision model supports image inputs when providing a single chat message. Example:
```bash npm2yarn
pnpm install @langchain/core
```
```typescript
import fs from "fs";
import { ChatGoogleGenerativeAI } from "@langchain/google-genai";
import { HumanMessage } from "@langchain/core/messages";
const vision = new ChatGoogleGenerativeAI({
model: "gemini-pro-vision",
maxOutputTokens: 2048,
});
const image = fs.readFileSync("./hotdog.jpg").toString("base64");
const input = [
new HumanMessage({
content: [
{
type: "text",
text: "Describe the following image.",
},
{
type: "image_url",
image_url: `data:image/png;base64,${image}`,
},
],
}),
];
const res = await vision.invoke(input);
```
The value of `image_url` can be any of the following:
- A public image URL
- An accessible gcs file (e.g., "gcs://path/to/file.png")
- A base64 encoded image (e.g., `data:image/png;base64,abcd124`)
- A PIL image
## Embeddings
This package also adds support for google's embeddings models.
```typescript
import { GoogleGenerativeAIEmbeddings } from "@langchain/google-genai";
import { TaskType } from "@google/generative-ai";
const embeddings = new GoogleGenerativeAIEmbeddings({
modelName: "embedding-001", // 768 dimensions
taskType: TaskType.RETRIEVAL_DOCUMENT,
title: "Document title",
});
const res = await embeddings.embedQuery("OK Google");
```
## Development
To develop the Google GenAI package, you'll need to follow these instructions:
### Install dependencies
```bash
pnpm install
```
### Build the package
```bash
pnpm build
```
Or from the repo root:
```bash
pnpm build --filter @langchain/google-genai
```
### Run tests
Test files should live within a `tests/` file in the `src/` folder. Unit tests should end in `.test.ts` and integration tests should
end in `.int.test.ts`:
```bash
$ pnpm test
$ pnpm test:int
```
### Lint & Format
Run the linter & formatter to ensure your code is up to standard:
```bash
pnpm lint && pnpm format
```
### Adding new entrypoints
If you add a new file to be exported, either import & re-export from `src/index.ts`, or add it to the `exports` field in the `package.json` file and run `pnpm build` to generate the new entrypoint.