Blog

Book a demo

MindsDB Change Log

Cover Image for MindsDB Change Log

MindsDB releases new features, improvements, and bug fixes weekly. Here you’ll find information on the latest product updates.

‍

Join our Slack community to interact with the core MindsDB team, ask questions, and share your thoughts.

Friday, September 15th, 2023

Features

We added the pgvector extension to the PostgreSQL handler that enables you to store vector embeddings and perform vector similarity search in PostgreSQL.

Improvements

  • Refactored the Writer handler to be able to work with different indexes and vector databases.

  • Fixed bugs related to creating and deleting chatbots.

Friday, September 8th, 2023

Features

Here are the features that cover vector databases:

  • Added a common approach for implementing vector databases.

  • Simplified the ChromaDB handler by removing dependency on LangChain and decoupling embeddings step.

The Gmail authentication process has been updated. Upon connecting to Gmail from MindsDB Cloud, users are redirected to a new window to authenticate their Gmail account.

MindsDB, as an AI workflow automation platform, enables all users to automate the data and prediction flow between their data sources and applications by disabling the restriction on job execution. All users, including the non-paid users, can execute an unlimited number of jobs.

Fixes

  • Added default port for MySQL connections.

  • Add an option to extend the default host and port for PostgreSQL connections.

  • Fixed the Couchbase bug to make table entries unique.

  • Fixed the LangChain parameter to not use the hardcoded value.

Wednesday, September 6th, 2023

Features

We already support bringing custom models to MindsDB. Now you can finetune your custom model within MindsDB.

The TimeGPT handler can now handle the finetune steps, token validation, and date features.

Fixes

As we grow, we continuously improve and refine the available features. Here are some fixes made in this release:

  • Removed obsolete and unused methods from models’ implementation.

  • Added an alert for missing training data to Lightwood.

  • Fixed LangChain prompt parameter name for conversational mode.

  • Fixed deprecated requirements, adding scikit-learn instead of sklearn.

  • Fixed batch predictions made using MongoDB.

  • Fixed the CREATE ML_ENGINE statement to work with empty arguments list.

  • Added support for non-grouped data in TimeGPT.

  • Fixed Hugging Face's pandas version.

  • Added Docker dependencies for MonkeyLearn.

  • Fixed Langchain error messages used to only display first 50 characters - now it displays the whole message.

  • TimeGPT fixes for frequency and levels.

  • Addeed missing requirements for the Autokeras handler.

  • Deleted mindsdb and mindsdb-sql from handlers' requirements.

  • Rounded timestaps in the Binance handler.

  • Predownloaded nltk data in Docker (punkt and stopwords).

  • Added Binance connector to Docker.

  • Fix to return new model record when calling train or finetune.

  • Fix for rare issue raised when integration data is empty.

  • Support to convert datecolumn to datetime in TimeGPT.

  • Removed restriction on using only SSH keys when cloning MindsDB repo.

  • Added DuckDB to Docker dependencies.

  • Added Plaid and Confluence dependencies to Docker.

Documentation

The aforementioned features, integrations, and bug fixes are updated in the MindsDB documentation.

Sunday, August 20th, 2023

Features

MindsDB supports the latest Python versions, including 3.10 and 3.11.

You can create triggers on the MongoDB data sources that allow you to execute predefined SQL code upon data updates.

We’ve got a new SQL syntax for chatbots: CREATE CHATBOT. It allows you to simply create a chatbot that includes the following features:

  • Chat pooling to reply promptly to messages,

  • Chat memory to navigate consistent conversations,

  • Chat executor that calls an AI model.

There is also a table that stores all chatbots and can be queried like this: SELECT * FROM chatbots.

The OpenAI integration offers an embedding mode. It allows you to get the predictions in the form of embeddings to improve the performance of the model, including its efficiency and accuracy.

You can connect MindsDB to SQL Alchemy, which we migrated from version 1.4 to 2.0.

You can not only connect models from the Hugging Face hub but also finetuned them in MindsDB. It includes Text classification, Zero shot text classification, Translation, and Summarization models available from MindsDB.

We improved the prediction process by running the tasks in a subprocess that prevents blocking the main process.

Integrations

We’ve implemented numerous integrations, including AI frameworks and applications.

  • The Replicate handler lets you use models available at Replicate, such as Audio Generation, img2txt, img2img, txt2img, and video AI.

  • The MonkeyLearn handler integrates this AI framework into MindsDB.

  • The Nixtla’s TimeGPT handler provides yet another model used for time-series forecasts.

  • The Webz handler integrates with the Webz API to make Webz data accessible from MindsDB.

  • The PyPI handler integration enables you to grab statistical data from PyPI and process it further with MindsDB.

  • The SentenceTransformers handler integrates this Python framework used for state-of-the-art sentence, text, and image embeddings.

Improvements

We strive to keep improving the features available at MindsDB. Here is a list of improvements made in this release:

  • Improved the CSV reading capabilities.

  • Enhanced the SELECT and DELETE methods of the Slack integration and improved error messages.

  • Updated the error message used when ML engines’ dependencies are missing.

  • Upgraded the Web Crawler integration to read PDF files provided in the form of URLs, and to fetch href values safely if href is empty. Also, improved error messages.

  • Optimized the PyPI handler.

  • Updated the requirements file used when installing MindsDB to include redis, fixing issues related to the redis cache.

  • Converted types to DuckDB-friendly types to improve type detection capabilities.

  • Fixed the Email handler to be able to SELECT emails with no issues.

  • Upgraded the StatsForecast handler to be able to create a model without the GROUP BY clause.

Bug Fixes

Thanks to our community, we are able to identify and fix bugs quickly. Here are some the bug fixes made in this release:

  • Fixed Shopify bugs in Docker.

  • Fixed bugs related to uploading files.

  • Removed convert_unicode for PostgreSQL database.

  • Fixed output when the DESCRIBE method is not implemented for an ML engine.

  • Fixed MonkeyLearn bugs.

  • Fixed StatsForecast bugs.

  • Fixed chatbot history limit to allow fetching conversation history.

  • Updated Snowflake and ClickHouse following the version update of SQL Alchemy.

  • Fixed the Apache Ignite connection issues, ensuring the port value is an integer, and added its dependencies to Docker.

  • Fixed Lightwood to pass the finetuning parameters.

  • Fixed TimeGPT to correctly convert date columns.

Documentation

The aforementioned features, integrations, and bug fixes are updated in the MindsDB documentation.

Thursday, August 3rd, 2023

Improvements and Updates

We have updated the MySQL handler configuration by adding the conn_attrs key, which is required for providing a partner name for the SingleStore data source.

The error messages have been improved for the following cases:

  • handler cannot be used,

  • removing handlers in list when they cannot be imported,

  • deleting ability to create integration via config file.

The documentation has been updated, including MS SQL Server, What is MindsDB, REST API, OpenAI, SurrealDB, Shopify, and Google BigQuery.

Fixes

  • Fixed the Writer handler by adding column names to meta data and an option to specify base_url directly.

  • Fixed type mismatch for models_versions table.

  • Added unknown argument validation for the OpenAI handler.

  • Fixed creating ML Engines without parameters for Mongo-QL.

  • Implemented a fix to not rise an exception when a table name has more than two parts (instead, send a query as is to an integration)

  • Fixes for Oracle connection include adding the disable_oob parameter to disable out-of-band breaks, the auth_mode parameter for connecting with database privileges, and fixing the get_table methods.

  • Fixed the Shopify integration dependencies by adding it to release file so it works with Docker.

  • Implemented a fix for file locking on some systems.

  • Fixed the StatsForcast ML engine on Docker.

Tuesday, July 25th, 2023

Improvements

We are working on the chatbot feature, so you can easily create custom chatbots based on your data. Currently, it is available on local MindsDB installation.

The integrations available at MindsDB are regularly updated to match the speed of developments in AI. We've decoupled the LangChain and OpenAI handlers since they don't share much functionalities.

Updates

We've made several updates to Python SDK. These include improving creation and deployment of time series models, enabling the predict method to take a dict data type as an argument, supporting file upload feature, supporting dropping tables, and updating the README file.

The new version of mindsdb_sql is available. It includes updates to the CREATE TRIGGER command and various fixes related to creating database and making predictions.

Integrations

Now you can use the Anthropic models within MindsDB. Find out more by following this link.

Fixes

  • Fixed the file upload mechanism to allow uploading CSV files with a single column.

  • Fixed the Dremio data lakehouse handler to enable SSL certificate verification.

  • Implemented fixes for the MindsDB's Docker image.

Wednesday, July 19th, 2023

Improvements

Lightwood, MindsDB's default AI engine, supports embedding mode, so you can return the predictions in the form of embeddings to speed up the process.

As MindsDB provides the RETRAIN and FINETUNE features, one model may have multiple versions. Now if you want to delete a model that has many versions, it is going to take less time thanks to the usage of threads.

MindsDB offers over 100 handlers, including data handlers and ML handlers. To see all available handlers, you can use the SHOW HANDLERS statement, defining the type as data or ml.

Lastly, the Frappe API handler was improved to show the error message.

Fixes

  • Fixed bug related to case-dependent project names. Now, the project names are case-independent, so "MyProject" is equivalent to "myproject".

  • Fixed the model status bug. The model status is set to error when training data cannot be obtained.

Tuesday, July 11th, 2023

Updates

MindsDB offers free access to all OpenAI models, with a slight change in the user verification process. In order to utilize the OpenAI models, users are now required to confirm their email address. To complete this verification, users need to navigate to the Settings section and click on the Confirm email button. By following the instructions outlined in the verification email, users can successfully confirm their email address and resume using the OpenAI models within MindsDB.

Integrations

MindsDB has introduced the LightFM and Popularity Recommender handlers that filter given data based on popularity.

You can learn more about the LightFM Recommender handler by following this link and about the Popularity Recommender handler by following this link.

Fixes

  • Fixed the Microsoft SQL Server handler to be able to SHOW TABLES present in the database.

  • Fixed an issue related to uploading files to MindsDB Cloud - now you can upload and query your files.

Friday, July 7th, 2023

Improvements

The process cache feature helps shorten the model training time. By starting processes that initialize the available ML handlers beforehand, we make them ready for the incoming tasks. So, when a user creates a model using the Lightwood ML engine, it picks up the process with the initialized Lightwood handler, assuming that such a process is ready and available.

MindsDB integrates with the Shopify application. We added support for the INSERT statement so that users can insert customer data into their Slack accounts.

Fixes

  • Fixed minor bugs in the Writer handler.

  • Fixed an issue related to the chatbot creation process to ensure the correct chatbot name.

  • Fixed an issue related to installing MindsDB via pip on Windows - now, we don’t import the fcntl package on Windows.

  • Fixed an issue related to certain data sources being undroppable. If you created a data source with a dot in its name, it wasn’t possible to drop it - this is now fixed.

  • Updated requirements for the LangChain handler.

Tuesday, June 27th, 2023

Chatbots

Currently, MindsDB offers Slack and Rocket Chat integrations for creating chatbots. When using local MindsDB installation (via pip or Docker) or a managed instance (MindsDB Pro), you can utilize the REST API endpoints to create and work with chatbots.

Here are the available REST API endpoints:

  • GET /projects/<project_name>/chatbots - gets all chatbots created by the user.

  • GET /projects/<project_name>/chatbots/<chatbot_name> - gets a chatbot by name.

  • PUT /projects/<project_name>/chatbots/<chatbot_name> - updates a chatbot with new settings, creating it if it doesn't exist.

  • POST /projects/<project_name>/chatbots - creates a new chatbot.

  • DELETE /projects/<project_name>/chatbots/<chatbot_name> - deletes a chatbot by name.

Additionally, we have developed a real-time Slack chat handler that implements an interface for sending and receiving Slack messages.

Integrations

Web Crawler

MindsDB has upgraded its capabilities with the introduction of web crawler integration, empowering users to retrieve valuable data from websites effortlessly. By integrating this functionality, users can now harness the power of MindsDB to extract and utilize website data within their AI-powered applications.

You can learn more about how to use it by following this link.

MediaWiki

MindsDB integrates with MediaWiki, providing users with the power to effortlessly query its pages and leverage their content in the development of AI-powered applications.

You can learn more about how to use it by following this link.

Improvements

MindsDB has enhanced its Gmail handler by introducing the UPDATE and DELETE methods, expanding its functionality beyond fetching and writing emails to include the capability to update and delete them as well.

We have recently made some updates to the Writer handler, addressing any missing dependencies and ensuring a seamless experience for users.

Fixes

  • Fixed an issue related to duplicated events in Google Calendar integration.

  • Lifted the limit on querying for batch predictions using OpenAI models.

  • Specified OpenAI models that can be fine-tuned within MindsDB. This follows the list of models as defined by OpenAI.

  • Fixed bugs related to the output of the DESCRIBE statement.

Tuesday, June 20th, 2023

The Llama Index handler has undergone a refactoring process to optimize its functionality and utilize appropriate abstractions. This refactoring is part of our ongoing efforts to enhance the handler, with more improvements planned for the future.

As a MindsDB user, you may already be aware that the presence of the target column is crucial in the training dataset. To ensure data integrity, we have implemented a check that raises an exception if the target column is missing from the training dataset. This enhancement aims to provide a more robust and error-free training process.

Thursday, June 15th, 2023

In this release, we have addressed the issues related to missing dependencies for the google_calendar, langchain, and llamaindex handlers.

Furthermore, the introduction of UPDATE and DELETE statements enables you to manipulate the data stored in your connected data sources.

We have also implemented a custom SQL function called json_extract(), which facilitates the extraction of JSON values by specifying a key from a JSON object.

In addition to these updates, we have made miscellaneous documentation improvements to enhance the overall user experience.

Monday, June 12th, 2023

Features

The PayPal handler has been updated to include support for querying payments.

Additionally, a new ML handler called Llama Index has been introduced, providing the ability to create models based on it.

Fixes

  • Improvements in memory and prompt templating for Langchain handler.

  • Completely removed the streams feature.

  • Fixes for keeping microservices up to date.

Documentation

  • Added Postgres API docs.

  • Updated Learning Hub tutorials.

Tuesday, May 23rd, 2023

Features

MindsDB released new exciting integrations handlers, including Rocket Chat, Frappe, and Shopify. Here are the details:

  • We implemented a basic Rocket Chat API handler that enables reading and writing messages to Rocket Chat channels.

  • The Frappe API handler is now available and capable of seamlessly inserting and retrieving documents.

  • MindsDB integrated the Shopify App handler that supports querying Products, Customers, and Orders.

Improvements

In pursuit of delivering the best quality product, MindsDB maintains a commitment to continuous improvement. Here are the latest enhancements:

  • Improved the FINETUNE statement, allowing finetune from particular model version, as well as finetuning without switching active version.

  • Improved debugging by replacing threading.get_ident with threading.get_native_id.

  • Fixed the get_table method in API Handler to use Identifier type.

  • Added dependencies for the Slack integration.

Documentation

MindsDB continuously enhances the user experience by consistently improving their content and documentation. Here are the latest updates:

  • Created Python SDK docs.

  • Created House Sales Tutorial Tests.

  • Created MindsDB Cloud docs on Demo, Starter, and Pro plans, adding instructions on how to connect to SQL and No-SQL clients.

  • Updated CREATE ML_ENGINE example to use OpenAI.

  • Created Bring Your Own Model docs.

  • Added minimum requirements for Docker docs.

  • Added ML_ENGINES commands to Mongo docs.

  • Updated docs navigation.

  • Updated supported Python versions to 3.8.x to 3.9.x.

  • Updated Google BigQuery docs.

  • Updated Google Calendar docs.

Wednesday, May 17th, 2023

General Updates

MindsDB uses the Lightwood ML engine by default unless specified differently in the CREATE MODEL statement. As such, we keep improving it. Now, the Lightwood engine is at version 23.5.1.0. Learn more by following the Lightwood docs.

We’ve also made updates to the LangChain ML engine.

The support for Python version 3.7 is dropped. Currently, MindsDB requires Python 3.8 or 3.9.

The REST API endpoints are now well-documented. You can use it to manage databases, tables, models, views, and more. Check out our REST API docs here.

We’ve created a cache for data handlers that keeps connections opened during TTL time from the handler's last use.

As MindsDB grows, we continuously update the documentation. We’ve made a major restructure to our docs, including dividing them into sections for better readability.

New Integrations with MindsDB

We’ve got a lot of new integrations coming up. Let’s look at the ones released this week.

GitLab

This integration enables organizations and users on GitLab to use ML for issue estimations, labeling, projects recommendation, automation of comments on GitLab issues, proofreading of READMEs, and more.

You can find out more about GitLab integration and its usage here.

Hugging Face Inference API

The Hugging Face Inference API lets you easily integrate NLP, audio, and computer vision models deployed for inference via simple API calls.

You can find out more about Hugging Face Inference API integration and its usage here.

Google Search Console

It enables MindsDB users to retrieve search results from Google directly within their ML models. Users can incorporate external data from Google Search to improve their model accuracy and predictions.

You can find out more about Google Search integration and its usage here.

Google Fit

This integration enables users to analyze their Google Fit data with the help of ML models in MindsDB.

You can find out more about Google Fit integration and its usage here.

Google Content API for Shopping

This integration enables users to leverage ML capabilities for sales predictions, inventory management, product recommendations, and other automation tasks.

You can find out more about Google Content API integration and its usage here.

Google Books

The Google Books integration with MindsDB enables users to use ML for book recommendations, reading history analysis, and other automation tasks based on their Google Books activity.

You can find out more about Google Books integration and its usage here.

Quickbooks

This integration enables Quickbooks users to use the power of machine learning for generating expense reports, risk assessments, accounts payable, and many other use cases.

You can find out more about Quickbooks integration and its usage here.

Sendinblue

This integration enables Sendinblue users to use ML for email campaign predictions, lead scoring, and other automation tasks in their Sendinblue account.

You can find out more about Sendinblue integration and its usage here.

Youtube

The integration of MindsDB and YouTube lets users query for videos’ comments and analyze them using ML models.

You can find out more about YouTube integration and its usage here.

Slack

With this integration, users can easily connect MindsDB to their Slack workspace and use powerful AI capabilities to enhance team communication and collaboration.

You can find out more about Slack integration and its usage here.

HackerNews API

This integration enables MindsDB users to gather data from HackerNews, including news articles, comments, and user profiles, for machine learning tasks such as sentiment analysis, content recommendation, and data analysis.

You can find out more about HackerNews integration and its usage here.

Strava

This integration enables users to access their Strava data, including activities, segments, and athlete information, through MindsDB, opening up possibilities for fitness-related automation, analysis, and insights.

You can find out more about Strava integration and its usage here.

NewsAPI Handler

This integration enables users to retrieve news articles from various sources, filter by keywords, and perform other operations using ML for news analysis, recommendation, and automation purposes.

You can find out more about NewsAPI integration and its usage here.

SAP MaxDB

This integration enables users of SAP MaxDB to connect it to MindsDB as a data source. It is a high-performance, scalable, and reliable relational database management system that supports a wide range of applications.

You can find out more about SAP MaxDB integration and its usage here.

Bug Fixes

  • Fixed output of the DESCRIBE…info command.

  • Fixed OpenAI integration in Docker image.

  • Added missing tweepy in Docker image.

  • Update the pyarrow version in Docker image.

  • Fixed issues with the ClickHouse handler.

  • Fixed the Google Calendar bugs.

  • Refactored time-series.

‍

Friday, Apr 28th, 2023

REST API Endpoints

We’ve got new REST API endpoints for databases and views.

‍

You can now fetch all databases connected to MindsDB using the GET /api/databases endpoint. We provide the POST and PUT endpoints to create or update a database. And to delete a database, you can call the DELETE /api/databases/<database_name> endpoint, passing your database name.

‍

The same goes for views. You can fetch, create, update, or delete them using the REST API endpoints. All these endpoints include a project name where the view resides. For example, to get a single view, you can call the GET /api/projects/<project_name>/views/<view_name> endpoint. To learn more about MindsDB projects, check out our docs here.

New Integrations with MindsDB

We’ve got a lot of new integrations coming up. Let’s look at the ones released this week.

Reddit

The integration with Reddit allows users to automate tasks, such as sentiment analysis, community trend analysis, and user behavior analysis, utilizing machine learning models.

Check out our docs here to learn how to connect your Reddit account to MindsDB.

‍

Plaid

Thanks to the integration with Plaid, users can apply ML for financial predictions, transaction categorization, and other financial automation tasks.

Check out our docs here to learn how to connect your Plaid account to MindsDB.

‍

Google Calendar

With the Google Calendar integration, you can now leverage the power of ML for smarter scheduling, event recommendations, and other automation tasks.

Check out our docs here to learn how to connect your Google Calendar to MindsDB.

‍

Confluence

The integration with Confluence enables users to create, collaborate, and organize their work, utilizing the power of ML.

Check out our docs here to learn how to connect your Confluence account to MindsDB.

‍

InfluxDB

MindsDB offers over 70 integrations with various data sources. Here is another one: InfluxDB is an open-source time series database that you can now connect to MindsDB.

Check out our docs here to learn how to connect your InfluxDB database to MindsDB.

‍

March 2023

We’ve implemented numerous new features, product improvements, and bug fixes. Read along to see the overview.

New Features and Product Improvements

  • Improvements to the StatsForecast engine:

  • Now users can select from ARIMA, CES, ETS, and Theta with the USING clause.

  • Users can manually specify the frequency of their dataset with the USING clause.

  • GPT4 model support for the OpenAI ML engine.

  • New database integrations:

  • New ML engine integrations (currently, in beta stage):

‍

Stay tuned for more exciting updates in the future!

Bug Fixes

  • Fixed error with handling nullable integers.

  • Fixed error with uploading .xlsx files.

  • Fixed issue with installing Trino handler dependencies.

  • Fixed issue with installing d0lt handler dependencies.

  • Fixed JOBS connection issues.

  • Fixed INSERT INTO issues when using Twitter integration.

  • Fixed issue with dependencies in Docker image.

  • Fixed issue with adjusting the OpenAI models.

February 2023

Here are the new features, product improvements, and bug fixes released this month.

New Features and Product Improvements

  • Deployment support for macOS with M1 chips.

  • Support for fine-tuning OpenAI models with new data and added new completion parameters.

  • New ML handler for StatsForecast models to increase the support for more time series libraries.

  • New database integrations:

  • New functionality for scheduling queries.

  • New integration with Twitter that provides automation to Twitter comments/posts with just two SQL commands.

  • New feature to show the model training progress.

  • New AutoKeras ML handler.

Bug Fixes

  • Fixed issue with time series models giving NULL results.

  • Fixed file upload issue.

  • Resolved issues with deleting ML engines.

‍

We would like to thank our community for ongoing support, and if you are new to MindsDB, please feel free to try it without installation using our free demo environment.