OLIY TAʻLIMDA GENERATIV AI AGENTLARI: XAVFSIZLIK CHEGARALARI VA ISHONCHLILIK
DOI:
https://doi.org/10.65164/98z68x49Ключевые слова:
generativ AI, katta til modeli (LLM) agentlari, prompt injection, maʻlumot qidirishga asoslangan generatsiya (RAG), kiberxavfsizlik, oliy taʻlim, xavfsiz arxitektura.Аннотация
Ushbu maqola generativ katta til modellari (LLM) asosidagi agentlarni oliy taʻlim muhitida xavfsiz joriy etishning asosiy arxitektura muammolarini koʻrib chiqadi. Biz uchta asosiy zaiflikni — prompt injection, kontekst chiqib ketishi va rol oʻzgartirishni — eksperimental tarzda sinab koʻrdik: 30 ta tizimlashtirilgan hujum stsenariysi yordamida himoyasiz va himoyalangan agent konfiguratsiyalarini solishtirdik. Natijalar shuni koʻrsatdiki, tizim koʻrsatmasi chegarasiga rioya qilish va retrieval-augmented generation (RAG) asosidagi bilim kapsulatsiyasi birgalikda hujum muvaffaqiyat darajasini 73% dan 13% gacha tushiradi. Shu asosda biz oliy taʻlimga moʻljalangan besh qatlamli xavfsiz agent arxitekturasini taklif etamiz.
Библиографические ссылки
[1] Bommasani, R., Hudson, D. A., et al. (2021). On the Opportunities and Risks of Foundation Models. arXiv:2108.07258.
[2] Anthropic. (2024). Claude Model Specification: Agentic and Multi-Agent Frameworks. Anthropic Technical Documentation. anthropic.com/docs.
[3] Perez, F., & Ribeiro, I. (2022). Ignore Previous Prompt: Attack Techniques for Language Models. NeurIPS Workshop on Trustworthy and Reliable Large-Scale Machine Learning Models.
[4] Greshake, K., Abdelnabi, S., Mishra, S., Endres, C., Holz, T., & Fritz, M. (2023). Not What You’ve Signed Up For: Compromising Real-World katta til modeli (LLM)-Integrated Applications with Indirect Prompt Injection. ACM CCS AISec Workshop.
[5] Lewis, P., Perez, E., Piktus, A., et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. Advances in Neural Information Processing Systems, 33, 9459–9474.
[6] Shi, W., Min, S., Yasunaga, M., et al. (2023). REPLUG: Retrieval-Augmented Black-Box Language Models. arXiv:2301.12652.
[7] OWASP Foundation. (2025). OWASP Top 10 for Large Language Model Applications, Version 2.0. Open Web Application Security Project.
[8] UNESCO. (2023). Guidance for Generative AI in Education and Research. United Nations Educational, Scientific and Cultural Organization (UNESCO), Paris.
724
[9] Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic Review of Research on Artificial Intelligence Applications in Higher Education. International Journal of Educational Technology in Higher Education, 16(1), 39.
[10] Oʻzbekiston Respublikasi. (2019). Shaxsga doir maʻlumotlar toʻgʻrisidagi Qonun. ZRU-547. Qonun hujjatlari milliy bazasi.
[11] Weidinger, L., Mellor, J., Rauh, M., et al. (2021). Ethical and Social Risks of Harm from Language Models. arXiv:2112.04359.
[12] Ross, R. S. (2013). Security and Privacy Controls for Federal Information Systems and Organizations. NIST Special Publication 800-53, Revision 4.
