Class ChatGoogleVertexAI

Enables calls to the Google Cloud's Vertex AI API to access Large Language Models in a chat-like fashion.

To use, you will need to have one of the following authentication methods in place:

  • You are logged into an account permitted to the Google Cloud project using Vertex AI.
  • You are running this on a machine using a service account permitted to the Google Cloud project using Vertex AI.
  • The GOOGLE_APPLICATION_CREDENTIALS environment variable is set to the path of a credentials file for a service account permitted to the Google Cloud project using Vertex AI.

Example

const model = new ChatGoogleVertexAI({
  temperature: 0.7,
});
const result = await model.invoke("What is the capital of France?");

Hierarchy

  • BaseChatGoogleVertexAI<GoogleAuthOptions>
    • ChatGoogleVertexAI

Constructors

constructor

Properties

connection examples maxOutputTokens model streamedConnection temperature topK topP

Methods

createInstance formatParameters convertPrediction convertPredictionChunk

Constructors

constructor

Properties

connection

connection: GoogleVertexAILLMConnection<BaseLanguageModelCallOptions, GoogleVertexAIChatInstance, GoogleVertexAIChatPrediction, GoogleAuthOptions<JSONClient>>

examples

examples: ChatExample[] = []

Help the model understand what an appropriate response is

maxOutputTokens

maxOutputTokens: number = 1024

Maximum number of tokens to generate in the completion.

model

model: string = "chat-bison"

Model to use

streamedConnection

streamedConnection: GoogleVertexAILLMConnection<BaseLanguageModelCallOptions, GoogleVertexAIChatInstance, GoogleVertexAIChatPrediction, GoogleAuthOptions<JSONClient>>

temperature

temperature: number = 0.2

Sampling temperature to use

topK

topK: number = 40

Top-k changes how the model selects tokens for output.

A top-k of 1 means the selected token is the most probable among all tokens in the model’s vocabulary (also called greedy decoding), while a top-k of 3 means that the next token is selected from among the 3 most probable tokens (using temperature).

topP

topP: number = 0.8

Top-p changes how the model selects tokens for output.

Tokens are selected from most probable to least until the sum of their probabilities equals the top-p value.

For example, if tokens A, B, and C have a probability of .3, .2, and .1 and the top-p value is .5, then the model will select either A or B as the next token (using temperature).

Methods

createInstance

formatParameters

StaticconvertPrediction

StaticconvertPredictionChunk

Settings

Member Visibility

On This Page

Constructors
Properties
Methods