Integration with Databricks
Introduction
Outdoo's platform analyzes your sales call data (recorded customer meetings) in real-time and deal data (pipeline and CRM opportunities) on scheduled intervals. Integrating this analyzed data into Databricks allows RevOps and data teams to unify conversation intelligence with other business data.
This allows deeper analysis of deal trends, rep performance, customer behavior, and pipeline risks within your existing BI tools and workflows. Connecting Outdoo to your warehouse lets you correlate call insights and AI-derived deal metrics with CRM, product usage, and revenue data for more informed decision-making.
Integration Overview and Supported Methods
Outdoo provides direct integration options to sync data to your warehouse without requiring third-party ETL tools. The following methods are supported:
REST API Pull: Outdoo offers RESTful APIs to retrieve call and deal data for custom integrations. This requires more custom development but provides flexibility if other methods do not meet requirements.
Cloud Storage Stage (S3 or ADLS): Outdoo can export data files in JSON format to a cloud storage bucket you own. Databricks can autoload files from cloud storage. This method is useful if direct database connections are not possible but may introduce slight delays (for example, daily batch file drops).
Security and Authentication Setup
Before starting, ensure you have prepared secure access for Outdoo:
- Create Dedicated Credentials: Provision a dedicated user or role in Databricks for Outdoo. Use a Service Principal with a token rather than a personal token for long-term integrations. Grant only the minimum privileges needed to create and write to Outdoo-specific schemas.
- Network Access: If your warehouse is behind a firewall or VPC, whitelist Outdoo's egress IP addresses or use a private connectivity method.
- API Keys and Tokens: Generate secure tokens for connections. Outdoo will securely store any tokens or secrets you provide.
- Certificate and Encryption: All integrations use encrypted channels (HTTPS for APIs, TLS for JDBC).
- Auditing and Revocation: Treat Outdoo integration credentials as you would any service account. Monitor activity and rotate keys/tokens periodically.
Databricks Integration
For Databricks (Lakehouse on AWS or Azure), Outdoo integrates via a direct write into your Databricks environment using the SQL Warehouse interface. Outdoo connects to a Databricks SQL Warehouse endpoint using JDBC or HTTP and creates data tables as Delta tables. This eliminates the need for intermediate files or separate ETL jobs.
Setup Steps
- Prepare a Databricks SQL Warehouse: In your Databricks workspace, create a new SQL Warehouse dedicated for integrations. Enable Unity Catalog if you use it. Outdoo can write to a specific catalog and schema you designate. Note the hostname and HTTP Path for the warehouse.
- Generate Access Token: Create a Personal Access Token in Databricks user settings, or create a dedicated Service Principal for Outdoo and generate a token for that principal. Ensure the token has permission to access the target warehouse and write to the target catalog/schema.
- Set Up Target Schema: Decide on a target Catalog and Schema in Databricks where Outdoo should write data (for example, the
maincatalog and a schema namedoutdoo_data). Ensure the Outdoo identity can CREATE TABLE in that schema.
- Provide Databricks Connection Info to Outdoo: In the Outdoo admin panel (Data Integration settings), select Databricks as a destination and enter the required fields: Server Hostname, HTTP Path, Port (usually 443), Catalog and Schema name, and the Access Token. Click "Connect" to authorize the integration.
- Data Sync Process: Upon successful connection, Outdoo creates Delta tables in the specified Databricks schema. Outdoo will batch-insert historical data first, then continuously push new call records after each call completes analysis. The DEALS table refreshes on a scheduled interval (for example, once per day). Outdoo uses upserts (MERGE) for tables where records may change, such as deal stage updates.
- Verify Tables in Databricks: Query the Delta tables to verify data. For example, run
SELECT count(*) FROM outdoo_data.calls;to see if the count matches the expected number of calls.
Schema and Data Considerations
The calls table contains columns such as call_id, call_datetime, participant_names, duration, sentiment_score, talk_ratio, and keywords. The deals table contains deal_id, deal_name, owner, stage, amount, close_date, last_call_date, and deal_risk_score. Outdoo may also create a DEAL_HISTORY or FORECAST_SUBMISSIONS table for historical tracking.
Tables will be Delta Lake tables, which support ACID updates. Outdoo may use a flag such as is_deleted for soft-deleted records.
With Outdoo data in Databricks, you can combine it with any other data in your Lakehouse. For example, a data science team might join the outdoo_data.calls table with product usage logs to see if call sentiment correlates with product adoption. An analytics team might build a BI dashboard that blends Outdoo's conversation insights with financial data. A machine learning pipeline might train on call transcripts combined with deal outcomes to predict deal success.