tMilvusInput
Retrieves records from a collection in the Milvus database and transfers them to the following component.
This component is not shipped with your Talend Studio by default. You need to install it using the Feature Manager. For more information, see Installing features using the Feature Manager.
tMilvusInput Standard properties
These properties are used to configure tMilvusInput running in the Standard Job framework.
The Standard tMilvusInput component belongs to the Databases family.
Basic settings
| Use existing connection |
Select this check box and in the Component List drop-down list, select the desired connection component to reuse the connection details you already defined. |
| Schema and Edit Schema |
A schema is a row description. It defines the number of fields (columns) to be processed and passed on to the next component. When you create a Spark Job, avoid the reserved word line when naming the fields. Click Edit schema to make changes to the schema. If the current schema is of the Repository type, three options are available:
|
| Collection | Enter the name of the collection in the Milvus database, or click the [...] button next to the corresponding field to fetch the available collections and select the desired one. |
| Operation | Select the type of operation you want to perform on the collection data:
If you are using dynamic fields to specify the collection, note that the QUERY mode will have a normal output, while the SEARCH mode will generate a new column called $meta with all the dynamic fields in it. |
| Query vector | This component supports vector search capabilities, you can use this option
to specify the query condition using the queryVector
property. This query vector should be a JSON array, or array of an array, like
[1,0,1,0,1] and
[[1,0,1],[1,0,1],[0,1,1]]. For more information on vector searches in Milvus, see the corresponding Milvus documentation. This field is available only when you have selected SEARCH from the Operation drop-down list. |
| Filter | Enter a query to filter on specific records in the collection data. For example, type in id < 50 to retrieve the records whose id is lower than 50 from the collection specified in the Collection field. |
| Limit and Offset |
For more information on these two parameters, see the corresponding Milvus documentation. |
Advanced settings
| tStatCatcher Statistics | Select this check box to collect the log data at the component level. |
| Partition | Select this check box to read data in partitions. Enter the name, in double quotation marks, of the partition column or columns of the Milvus table you want to read data from, or click the [...] button next to the corresponding field to select the desired ones. If you need to list several values, separate them with a comma. |
| Load collection into memory | Select this check box to load the selected collection into memory when executing the Job. |
Global Variables
| Global Variables |
NB_LINE: the number of rows read by an input component or transferred to an output component. This is an After variable and it returns an integer. ERROR_MESSAGE: the error message generated by the component when an error occurs. This is an After variable and it returns a string. This variable functions only if the Die on error check box is cleared, if the component has this check box. A Flow variable functions during the execution of a component while an After variable functions after the execution of the component. To fill up a field or expression with a variable, press Ctrl+Space to access the variable list and choose the variable to use from it. For more information about variables, see Using contexts and variables. |
Usage
| Usage rule |
|