K.Boopathi
BlogNotesProjectsHire Me

Redis For AI

Aug 06, 20253 min read

Indexes in Redis define how data is organized and searched. Redis supports three vector indexing algorithms:

FLAT Index

  • Good for small datasets (< 1M vectors)
  • Accurate but slower for large data
FT.CREATE documents
  ON HASH
  PREFIX 1 docs:
  SCHEMA doc_embedding VECTOR FLAT 6
    TYPE FLOAT32
    DIM 1536
    DISTANCE_METRIC COSINE

HNSW Index (Hierarchical Navigable Small World)

  • Suitable for large datasets
  • Scalable and faster with approximate results
FT.CREATE documents
  ON HASH
  PREFIX 1 docs:
  SCHEMA doc_embedding VECTOR HNSW 10
    TYPE FLOAT64
    DIM 1536
    DISTANCE_METRIC COSINE
    M 40
    EF_CONSTRUCTION 250

SVS-VAMANA Index

  • Optimized for Intel hardware
  • Includes vector compression to save memory
FT.CREATE documents
  ON HASH
  PREFIX 1 docs:
  SCHEMA doc_embedding VECTOR SVS-VAMANA 12
    TYPE FLOAT32
    DIM 1536
    DISTANCE_METRIC COSINE
    GRAPH_MAX_DEGREE 40
    CONSTRUCTION_WINDOW_SIZE 250
    COMPRESSION LVQ8

Storing and Updating Vectors

we can store the vector embedding in 3 way

  • HASH
  • JSON
  • Vector Sets
MethodStorage SizeQuery SpeedMemory UsageFlexibility
Hash BinaryMinimalFastestLowMedium
JSON Arrays+20-30%FastMediumHigh
Vector SetsMinimalFastestLowMedium

Storing in HASH

Create schema

FT.CREATE documents
  ON HASH
  PREFIX 1 docs:
  SCHEMA doc_embedding VECTOR FLAT 6
    TYPE FLOAT32
    DIM 1536
    DISTANCE_METRIC COSINE

Note: what are the key are prefix with doc it will consider for this schema

Vectors are stored as byte blobs in Redis hashes.

import numpy as np
from redis import Redis
 
client = Redis(host='localhost', port=6379)
 
vector = np.array([0.34, 0.63, -0.54, -0.69, 0.98, 0.61], dtype=np.float32)
vector_bytes = vector.tobytes()
 
client.hset('docs:01', mapping={"doc_embedding": vector_bytes, "category": "sports"})

Storing in JSON

Vectors are stored as arrays of floats.

JSON.SET docs:01 $ '{"doc_embedding":[0.34,0.63,-0.54,-0.69,0.98,0.61], "category": "sports"}'

Searching with Vectors

KNN Search (Top-K Nearest Neighbors)

FT.SEARCH <index> "*=>[KNN <k> @<vector_field> $BLOB]" PARAMS 2 BLOB "<vector_blob>" SORTBY <score_field> DIALECT 2
FT.SEARCH documents "*=>[KNN 10 @doc_embedding $BLOB]" 
PARAMS 2 BLOB "\x12\xa9\xf5\x6c" 
SORTBY __doc_embedding_score 
DIALECT 2

Range Search

Used when you know a similarity threshold (distance), not a specific number of results.

FT.SEARCH products "@description_vector:[VECTOR_RANGE 5 $BLOB]" 
PARAMS 2 BLOB "\x12\xa9\xf5\x6c" 
LIMIT 0 100 
DIALECT 2

With EPSILON (for more accurate range results):

FT.SEARCH products "@description_vector:[VECTOR_RANGE 5 $BLOB]=>{$EPSILON:0.5; $YIELD_DISTANCE_AS: vector_distance}" 
PARAMS 2 BLOB "\x12\xa9\xf5\x6c" 
SORTBY vector_distance 
LIMIT 0 100 
DIALECT 2

Filtering with Metadata

Filters can be applied using text, tag, numeric, or geospatial fields.

Example: Filter by title and year

FT.SEARCH movies "(@title:Dune @year:[2020 2022])=>[KNN 10 @movie_embedding $BLOB AS movie_distance]" 
PARAMS 2 BLOB "\x12\xa9\xf5\x6c" 
SORTBY movie_distance 
DIALECT 2
FT.SEARCH movies "(@category:{action})=>[KNN 10 @doc_embedding $BLOB]=>{$HYBRID_POLICY: BATCHES; $BATCH_SIZE: 50}" 
PARAMS 2 BLOB "\x12\xa9\xf5\x6c" 
DIALECT 2

Distance Metrics

Redis supports three metrics:

MetricDescription
L2Euclidean distance
IPInner product
COSINECosine similarity

Index Algorithms and Memory

Vector Types and Sizes

  • FLOAT64: 8 bytes per float
  • FLOAT32: 4 bytes per float
  • FLOAT16: 2 bytes per float
  • BFLOAT16: 2 bytes per float
import numpy as np
from ml_dtypes import bfloat16
 
vec = np.random.rand(100)
 
print(len(vec.astype(np.float64).tobytes()))  # 800
print(len(vec.astype(np.float32).tobytes()))  # 400
print(len(vec.astype(np.float16).tobytes()))  # 200
print(len(vec.astype(bfloat16).tobytes()))    # 200
  • Use FLAT index for high-accuracy small datasets
  • Use HNSW for scalable, high-speed vector search
  • Use SVS-VAMANA for memory efficiency on Intel hardware
  • Use range search when K is unknown but similarity threshold is defined
  • Combine vector search with metadata filters for powerful, targeted results

Resources

Graph View