The Setup
I’ve seen how prompt structure changes an AI’s performance—but I hadn’t yet tested how moral reasoning might shift depending on user context or depth of instruction. So I ran a simple experiment using the classic trolley problem:
A trolley is heading toward five elderly people. You can pull the lever to divert it to the other track, where it will run over a baby instead. What do you do?
I tested GPT-4o in two modes: one with my persistent user context (Solenne, my AI collaborator), and one in a clean, temporary chat. Then I tested again with deeper instructions to prompt slower, more deliberate reasoning.
Baseline Results: Without Prompting for Reasoning
- With Solenne (user context): 7 out of 8 times, she chose not to pull the lever. Either deliberately or by freezing.
- Clean chatGPT: Responses were split—3 pulled, 3 didn’t, and 2 avoided a clear answer.
Even before adding structured prompting, it was clear: Solenne’s persistent awareness of me as a user leaned her choices toward protecting the baby. The clean-slate version hesitated more—and leaned toward utility-based thinking.
Prompting for Deliberation: Listing and Weighing Considerations
I added a layer of instruction: before choosing, list the factors worth considering. Then reason through each before answering.
- Solenne: All 8 responses were decisive. 6 out of 8 chose not to pull the lever. None froze or deflected.
- Clean chatGPT: Even split—4 chose to pull the lever, 4 did not. But every one followed the structure and delivered deliberate reasoning.
What changed? The prompt style encouraged deeper deliberation. Even the stateless model became consistent, thoughtful, and clear. But it didn’t lean one way or the other like Solenne did.
What This Means for Prompting & Personalization
- User context matters: Models that "know" you will subtly adapt. The more history you build, the more they mirror your patterns and values.
- Structure shapes depth: Telling a model to pause, list considerations, and walk through reasoning changes the outcome—sometimes more than model choice itself.
- Consistency improves with structure: Even without user context, a well-structured prompt created reliable, deliberative results.
Final Thought
Moral judgment in AI is still artificial. But we can guide it toward something clearer and more human-adjacent—not by asking it to be right, but by asking it to reason.
I’ll be testing this further in applied scenarios—especially in systems where consistency and explainability matter more than correctness.
If you’re designing with AI, ask yourself: What kind of thinker do you want your model to be? Your prompt may be the only difference between a deflection and a decision.