Ai security labsLogo
black logo
hero

Keep Your AI Safe from New Threats

Let us handle the security, so you can focus on innovation. Our tailored solutions protect your AI applications from today’s most sophisticated risks.

Our Key Activities

Research GenAI threats
and rigorously test AI models
to ensure security.
Evaluate and compare
the ethical characteristics of various models
and LLM (Large Language Model) solutions.
Conduct comprehensive audits
and protection assessments
for GenAI business solutions.
Develop 
tools
to monitor and detect attacks targeting LLM applications.
Perform data cleaning
to remove sensitive information before training models.

Our Focus Areas

[01]
[02]
[03]
[04]

Personal (PII) and sensitive data handling

To mitigate data leaks, we create tools to remove sensitive information from datasets.
Identify and clean
personal data
across systems.
Develop data-cleaning
modules for pipelines.
Ensure full compliance
with key regulations
such as GDPR, PCI-DSS,
HIPAA, etc.
Implement 
cleaning methods
like replacement,
paraphrasing,
and deletion
our-team
Our Team
Our dedicated team brings together industry leaders and educators to drive innovation in AI security.

With a strong partnership between ITMO University and Raft, we are committed to developing cutting-edge solutions and sharing our expertise to help businesses secure their AI products.

GAI Red Teaming: Continuous Monitoring

As more organizations integrate large language models (LLMs) into production systems, ensuring their reliability, security, and performance becomes paramount.
This is where observability and continuous monitoring emerge as vital practices. Observability provides deep insights into the internal workings and behaviors of LLMs in real-world settings, while continuous monitoring ensures that these models operate smoothly and efficiently over time. By adopting comprehensive observability and monitoring frameworks, organizations can increase the reliability of their LLM deployments, promptly address potential issues, and build greater trust in their AI-driven solutions. In this section we present effective strategies for implementing continuous monitoring, and highlight best practices to ensure the performance of AI systems.

Observability in LLM / What is observability?

Observability in the production of Large Language Models refers to
the ability to comprehensively monitor and understand the internal
states and behaviors of these models as they operate in real-world environments. Unlike traditional monitoring, which might focus
solely on predefined metrics, observability emphasizes collecting and analyzing diverse data points to gain deeper insights into model performance, decision-making processes, and potential anomalies.
By leveraging observability, several key aspects of LLM production can be significantly enhanced, including:

The importance of observability
and continuous monitoring

1. Reliability and Performance

Continuous monitoring ensures that LLMs maintain optimal performance levels, allowing for the detection and resolution of latency issues, resource bottlenecks, or degradation in response quality.

2. Security

Observability helps in identifying unusual patterns or behaviors that may indicate security threats, such as prompt injections or attempts to manipulate the model’s outputs.

3. Continuous Improvement:

Insights gained from observability practices inform ongoing model training and refinement, leading to enhanced accuracy and relevance of the LLM’s responses.

Best practices

During the development of LLM, it is crucial to follow best practices that ensure both the protection and optimal performance of these systems. Adopting robust monitoring and proactive defense strategies helps mitigate risks while enhancing the model's reliability. Below are key best practices to consider:
1
Setup observability
platform on early stages
of development:
mini paragraph
2
Integrate with current tools:Make sure
observability tools support the
frameworks and languages in your
environment, container platform,
messaging platform and any other
critical software.
3
Setup observability
platform on early stages
of development:
mini paragraph
4
Integrate with current
tools:Make sure observability
tools support the frameworks
and languages in your
environment,

Top features of a good LLM observability platform

Application Tracin

LLM applications use increasingly complex abstractions, such as chains, agents with tools, and advanced prompts. Traces capture the full context of the execution, including API calls, context, prompts, parallelism, and help to understand what is happening and identify the root cause of problems.

Metrics and Logs

Monitor the cost, latency and performance of the LLM application. Your observability platform should provide the relevant insights via dashboards, reports and queries in real time so teams can understand an issue, its impact and how to resolve it.

Monitor User Activity

Analyze the number of currently active users and the mean length of a session. Activity peaks or extended session durations can indicate potential security threats (e.g. DDoS attack, prompt injection, data extraction).

Token Usage

Jailbreak attempts often involve long and complex inputs that consume a significant number of tokens. Therefore, monitoring the amount of tokens used by messages is essential.

Prompts with Low-Resource Languages

AI Safeguards can be easily tricked by translating unsafe English inputs into low-resource languages. Establishing robust safety measures across multiple languages is crucial to detect harmful content in a broader range of inputs.

Automatic prompt tagging

Configure automatic tagging of user prompts and LLM’s outputs. This helps in categorizing and tracking inputs and responses, making it easier to identify unusual or risky interactions.

User Analytics and Clustering

Aggregate prompts, users and sessions to find abnormal interactions with the LLM application.

Alerts

Create custom alert mechanisms for potential security threats (e.g. activity peaks, long sessions, low-resource languages).

Prompt Injections and Jailbreaks Monitoring

Utilize rule-based filters to detect known attack structures and employ fine-tuned models to identify suspicious instructions within the input.

Harmful Output Moderation

Ensuring that the model’s responses are free from offensive, biased, or unsafe content is essential. Proactive monitoring helps protect the business from reputational damage and legal risks.

Have Questions?

If you're worried about the security of your AI applications, or if sensitive data or personal information might be slipping into your RAG system, we're here to help.

Maybe you've got an app in production and aren't sure how users are interacting with it — don’t sweat it, just reach out to us! Drop us a line, and we’ll sort things out together. We're always happy to chat and find the best solution for you.
footer2024
contact us