This document covers all aspects of data storage, persistence patterns, and VBase integration in the VTEX Rewriter application.
📋 Table of Contents
- VBase Storage Structure
- Storage Management
- VBase Operations
- Reverse Index Management
- Validation Patterns
💾 VBase Storage Structure
💾 Storage Management
VBase Operations
_33class Routes<T> {_33 async get(locator: RouteLocator): Promise<T | null> {_33 const key = this.toFilesystemPath(locator)_33 _33 try {_33 const data = await this.vbase.getJSON(this.bucket, key)_33 _33 // Validate expiration date and other criteria_33 if (!this.isValid(data)) {_33 await this.delete(locator)_33 return null_33 }_33 _33 return data_33 } catch (error) {_33 if (error.status === 404) return null_33 throw error_33 }_33 }_33 _33 async save(route: T): Promise<void> {_33 const key = this.toFilesystemPath(route)_33 await this.vbase.saveJSON(this.bucket, key, route)_33 _33 // Invalidate local cache_33 this.cache.delete(key)_33 }_33 _33 private toFilesystemPath(locator: RouteLocator): string {_33 const encoded = encodeURIComponent(locator.from)_33 return `${locator.binding}/${encoded}.json`_33 }_33}
Reverse Index (Internals)
_18class RoutesByBindingsByEntity {_18 // Maps entity -> binding -> route_18 // Example: product/123 -> { "binding1": "/produto/abc", "binding2": "/product/abc" }_18 _18 async load(route: Internal): Promise<Record<string, string>> {_18 const key = this.getEntityKey(route)_18 return this.vbase.getJSON(this.bucket, key) || {}_18 }_18 _18 async save(route: Internal, bindings: Record<string, string>): Promise<void> {_18 const key = this.getEntityKey(route)_18 await this.vbase.saveJSON(this.bucket, key, bindings)_18 }_18 _18 private getEntityKey(route: Internal): string {_18 return `${route.type}/${route.id}.json`_18 }_18}
📊 Validation Patterns
Load-time Validation
_18const validationOnLoad = (routesVersion: number) => (_18 internal: Internal_18): boolean => {_18 const originInBlacklist =_18 internal.origin && INTERNAL_ORIGIN_BLACKLST.has(internal.origin)_18 const invalidRoutesVersion =_18 internal.type !== USER_ROUTE_TYPE &&_18 internal.routesVersion !== routesVersion_18 if (_18 originInBlacklist ||_18 invalidRoutesVersion ||_18 !internal.binding ||_18 !validDate(internal.endDate)_18 ) {_18 return false_18 }_18 return true_18}
Save-time Validation
_21const validationOnSave = async (_21 _: Routes<Internal>,_21 routes: Internal[]_21): Promise<boolean> => {_21 // ensure there are not overlaps in saveMany routes_21 if (routes.length > 1) {_21 const routeLocators = new Set<string>()_21 for (const route of routes) {_21 const locator = `binding=${route.binding}__from=${normalizePath(_21 route.from_21 )}`_21 if (routeLocators.has(locator)) {_21 throw new Error(_21 `Route {from: ${route.from}, binding: ${route.binding}} is duplicated in the batch`_21 )_21 }_21 routeLocators.add(locator)_21 }_21 }_21 return true_21}
🚀 Auto-Cache Storage Pattern
The auto-cache system introduces a new storage pattern for automatically saving compatibility routes to VBase.
Background Save Operations
_22// Auto-cache storage pattern - non-blocking saves_22const saveCompatibilityRouteInBackground = (_22 compatibility: Internal,_22 internals: any,_22 redirects: any_22) => {_22 const tomorrow = new Date()_22 tomorrow.setDate(tomorrow.getDate() + 1)_22 _22 const routeWithExpiry = {_22 ...compatibility,_22 endDate: tomorrow.toISOString(), // 24h TTL_22 }_22 _22 // Fire-and-forget save - doesn't block user response_22 setImmediate(() => {_22 internals.save(routeWithExpiry, redirects).catch((error) => {_22 // Silent error handling for background operations_22 console.warn('Auto-cache save failed:', error.message)_22 })_22 })_22}
Auto-Cache Storage Flow
Storage Performance Metrics
Traditional Save Operations:
- Blocking: Yes (waits for VBase response)
- User Impact: Added latency during save
- Error Handling: Must handle in request context
Auto-Cache Save Operations:
- Blocking: No (background with setImmediate)
- User Impact: Zero - immediate response
- Error Handling: Silent/logged only
- TTL: 24 hours automatic expiration
- Workspace: Production-only (master)
VBase Auto-Cache Keys
Auto-cached routes follow the same VBase key structure but are populated automatically:
_10VBase Bucket: internals_v2_10Key Format: {binding-id}/%2F{url-path}.json_10_10Examples:_10- aacb06b3-a8fa-4bab-b5bd-2d654d20dcd8/%2Fproduto%2Fsmartphone%2Fp.json_10- aacb06b3-a8fa-4bab-b5bd-2d654d20dcd8/%2Fcategoria%2Feletronicos.json
This storage pattern ensures that popular routes automatically become fast VBase hits while maintaining system responsiveness.
_10