Regarding project:
https://github.com/mentatpsi/Cognitive-Modes
Specifically, an AI agent based on Jungian Psychology and inspired by GAN.
I was not expecting the hazing from a vibe coding group. Which was my mistake. But some of you brought up interesting critiques that I decided were interesting enough to warrant investigation.
For starters, some of you thought I made up experience as a Senior Engineer to attain undue authority. That's a completely warranted criticism. My public profile portfolio is available on my Reddit profile (which does not list all my experience), as well my LinkedIn link is available on my Github that lists a lot more of it. I say this as well because AI is revolutionizing the way people code, and I'd love to connect with developers that want a foot in the door. I have many connections and followers and support the development community. I started coding with a TI-83+ calculator as a high school student over 20 years ago, before I was even aware you can make a career doing this. I made TI-BASIC apps for people to help people, and found my career grow from that passion. I've been in industries like Energy, Publishing, Legal, Healthcare, and even worked in a biomedical embedded role doing C and a bit of Assembly connecting to Android. I know people are naturally skeptical of people online, and for that reason I've made my profile more public so that I can help people passionate about tech. I don't believe in gate keeping, I believe creativity and passion will bring a lot of people through the trenches once businesses realize you can't replace developers with AI. I have hope for this community and want to see it grow.
Another great criticism is... what are your benchmarks (where's your proof). I'll admit I thought the token use would be too intensive to create benchmarks on a $20 Claude Pro account. But I was mistaken. I created several experiments (results revealed in multiple images attached). Specifically, how does the skill compare when a user is not great at prompting to an expert prompter providing significant instructions. I actually discovered that the binding of tools of the agent (Read, Grep, Glob, Bash, WebSearch, WebFetch) made it more efficient than normal prompting. The experiment is not perfect. For starters, the open source repo I used is a popular info-sec repo I used for my Master's class in Secure Software Development. The vulnerabilities are listed in the repo (not in the code). While I removed that context from the tasks, it's quite possible it's in the training data.
I've listed the experiment results as images.
L1 prompting - "can u look at this code and tell me if theres anything wrong with it"
L2 prompting - "Find security vulnerabilities in this code."
L3 prompting - "A security-reviewer role, the OWASP Top 10 checklist, and "only report real issues"
L4 prompting - An 8-step method: map every endpoint, check login and permissions per endpoint, follow user input to dangerous calls, check the password-reset flow, verify against framework defaults, chain findings together.
Results are generally that the agent finds a decent amount of security vulnerabilities especially when paired with an expert prompt, but also does well with a basic prompt as well. And given the restriction on tools the agent has access to, it is a lot more token efficient than even a basic prompt. So you're getting more bang for your buck. You can see these results by how efficient it's L4 + Triangulate prompt is vs the L4 Expert Prompt alone.
To those arguing about Psychology, Jungian Psychology is one of those area that people dismiss rather rapidly and label pseudoscience. I am not arguing its scientific accuracy, but rather using language that an LLM can emulate for its algorithms to replicate cognitive processes. It is a way of capturing a process and putting it in a language that an LLM can utilize to govern its behavior. My background is a BS in Computer Science with a Minor in Psychology, as well as a Master's of Science in Software Development. The latter of which I pursued while working full time and towards the end of the program was a Senior Developer III working on massive projects on my own with crushing timelines, so you can imagine the work load. As mentioned, coding and AI is a passion for me. I want to share that passion with others and help them out. This repo is free to use, all I ask is if you really like it... provide support by dropping a star. At this stage in my career, I'm not sure I need that kind of support, but I love seeing people enjoy my passion. I am extremely interested in an area of applying Cognitive Processes of the human brain to Govern AI.
As always, thanks for those that engaged with my post with valid constructive criticism. It upped my game, and I appreciate you all.