Freedom is still necessary but it must be adapted to the times.”

:pushpin:If one day AI has a protector and a destroyer in the same body.:interrobang::bomb::pray:

Hello everyone.:pray:
Generative AI has been the talk of the town for over 18 months now, with companies recognizing its value and therefore generative AI budgets continuing to grow. :tada:We expect more and more AI agents to be available in the coming years. :pushpin:It’s imperative that we examine how consumers interact with generative AI agents. :pushpin:How do developers build AI agents into their apps?
Identity :pushpin: remains the most common attack vector used by cybercriminals :roll_eyes: whether it’s phishing :roll_eyes:, impersonation or password spraying :interrobang:
:boom: 75% of cybersecurity professionals would say current threats are the biggest challenge they’ve seen in the past 5 years :pushpin: But cybersecurity workforce cuts and a widening skills gap are posing challenges to the industry :interrobang::boom:

:pushpin:Some of the problems encountered may be due to insufficient freedom.
(:point_right:The sample code is reserved. If interested, please ask for more details.:pray:)
:boom:Example :pushpin:TrojAiH (Troj… Ai Hybrid) may be considered similar to trojai, it is a Python module used to create a called dataset and a deep learning model of the Trojan (but if it is added with 6 more features):roll_eyes: This module has two submodules: TjAiH.datagen and TjAiH.modelgen.
Typically, an AI with a Trojan should continue to exhibit normal behavior for non-triggered inputs, lest it alert the user. Finally, :scream: a trigger will be most useful to an adversary if it is something that can be controlled in the AI’s operating environment :pushpin: so that they can intentionally activate the Trojan’s behavior​:interrobang: On the other hand, a trigger is something that exists naturally in the world​:loudspeaker: but only exists sometimes when the adversary knows what he wants the AI to do. :pushpin::pushpin: The specifics of a Trojan attack set it apart from the more common category of “data poisoning attacks,” in which the adversary distorts the AI’s training data to make it ineffective.
:boom: Clear defenses against Trojan attacks include
:pushpin: Securing the training data (to prevent data from being altered)
:pushpin: Cleaning the training data (to ensure the validity of the training data)
:pushpin: Protecting the integrity of the trained model (Prevent further malicious manipulation of clean trained models) :scream::roll_eyes::boom:Unfortunately, the advancements of modern AI are characterized by​:bomb: a large number of datasets collected from a large number of sources (e.g. the 1e9 data point):boom::bomb: which cannot be truly cleaned or verified​:interrobang::interrobang::interrobang: In addition, many specialized AIs are built through transfer learning, i.e.:boom: taking existing AI published online and
:roll_eyes: slightly modifying it for new use cases.
:cold_face::loudspeaker: Trojan behaviors can persist in these AI after modification. Therefore, AI security depends on the security of the data and the overall training process, which may be weak or non-existent, or
:boom: perhaps modern users may not perform any training at all. :face_with_spiral_eyes: Users can obtain AI from vendors or​:boom::pushpin::pushpin::pushpin: open model repositories that are malicious,:boom: compromised, or inefficient. Obtaining AI from elsewhere raises data and workflow security issues, including the possibility that AI is directly modified while stored at the vendor or during delivery to users​:bomb:
:pushpin::pray:No matter what the majority opinion is :pray: I respect the general consensus and the safety stance is the same as the first day I joined :pray: without any bias.
One vulnerability inspector / Many vulnerabilities. :cold_face: But cybercriminals only want one error." There is no equality.:pray:

Thank you for your thought-provoking post on security in AI, with the technology being used by both attackers and defenders.

I think you will be interested in my recent article on Turning Open Source AI Insecurity up to 11 with OSI’s RC1, in which I make the case for the data being required for Open Source AI on the basis that it is only with the data that you have a chance of finding the needle in the haystack that may compromise your security.

The OWASP Top 10 for Large Language Model Applications was working its way through final review at the time so there were 11, and the OSAID release candidate enabled or exacerbated all of them!

Additionally, you need the data to assess and address ethical issues including bias and fairness, and for the models to be able to form the foundation for future generations.

Welcome to the discussions and I look forward to hearing more from you in future.