←back to thread

427 points JumpCrisscross | 6 comments | | HN request time: 0s | source | bottom
Show context
lwhi ◴[] No.41901852[source]
It is no longer effective to solely use a written essay to measure how deeply a student comprehends a subject.

AI is here to stay; new methods should be used to assess student performance.

I remember being told at school, that we weren't allowed to use calculators in exams. The line provided by teachers was that we could never rely on having a calculator when we need it most—obviously there's irony associated with having 'calculators' in our pockets 24/7 now.

We need to accept that the world has changed; I only hope that we get to decide how society responds to that change together .. rather than have it forced upon us.

replies(26): >>41902001 #>>41902004 #>>41902006 #>>41902027 #>>41902041 #>>41902094 #>>41902144 #>>41902281 #>>41902432 #>>41902446 #>>41902471 #>>41902612 #>>41902683 #>>41902805 #>>41902892 #>>41903019 #>>41903144 #>>41903279 #>>41903529 #>>41903547 #>>41903572 #>>41903881 #>>41904424 #>>41904494 #>>41904546 #>>41905807 #
strogonoff ◴[] No.41902027[source]
The best method for assessing performance when learning is as old as the world: assess the effort, not how well the result complies with some requirements.

If the level of effort made is high, but the outcome does not comply in some way, praise is due. If the outcome complies, but the level of effort is low, there is no reason for praise (what are you praising? mere compliance?) and you must have set a wrong bar.

Not doing this fosters people with mental issues such as rejection anxiety, perfectionism, narcissism, defeatism, etc. If you got good grades at school with little actual effort and the constant praise for that formed your identity, you may be in for a bad time in adulthood.

Teacher’s job is to determine the appropriate bar, estimate the level of effort, and to help shape the effort applied in a way that it improves the skill in question and the more general meta skill of learning.

The issue of judging by the outcome is prevalent in some (or all) school systems, so we can say LLMs are mostly orthogonal to that.

However, even if that issue was addressed, in a number of skills the mere availability of ML-based generative tools makes it impossible to estimate the level of actual effort and to set the appropriate bar, and I do not see how it can be worked around. It’s yet another negative consequence of making the sacred process of producing an amalgamation of other people’s work—something we all do all the time; passing it through the lens of our consciousness is perhaps one of the core activities that make us human—to become available as a service.

replies(4): >>41902126 #>>41902155 #>>41902832 #>>41903663 #
1. injidup ◴[] No.41902155[source]
Little Johnny who tried really hard but still can barely write a for loop doesn't deserve a place in a comp sci course ahead of little Timmy who for some reason thinks in computer code. Timmy might be a lazy arse but he's good at what he does and for minimal effort the outcomes are amazing. Johnny unfortunately just doesn't get it. He's wanted to be a programmer ever since he saw the movie Hackers but his brain just doesn't work that way. How to evaluate this situation? Ability or effort?
replies(2): >>41902359 #>>41903117 #
2. strogonoff ◴[] No.41902359[source]
My evaluation:

1. Whoever determined that he does not “deserve” this is wrong. There may be other constraints, but no one gets to frame it as “deserves” when a child wants to learn something.

2. If a teacher is unable to teach Johnny to write a for loop, despite Johnny’s genuine utmost motivation, I would question teacher’s competence or at least fit.

3. Like any mentor, a professor in higher ed may want to choose whom to teach so that own expertise and teaching ability is realized to the fullest. Earlier in life, elementary school teacher’s luxury to do so may be limited (which is why their job is so difficult and hopefully well-compensated), and one bailing on a kid due to lack of patience or teaching competence is detestable.

4. If Johnny continues to pursue this with genuine utmost motivation, he will most likely succeed despite any incompetent teachers. If he does not succeed and yet continues to pursue this to the detriment to his life, that is something a psychologist should help him with.

As for Timmy, if he learns to produce the expected result with least effort, for which he receives constant praise from the teacher, and keeps coasting this way, that does him a major disservice as far as mental mental and self-actualisation in life.

replies(1): >>41905674 #
3. Skeime ◴[] No.41903117[source]
The evaluation criteria don't need to be the same for your entire life. So if someone is taking an exam to decide whether they're fit to become a bridge engineer, ability should be the criterion. Little Johnny in school can still be evaluated based on effort. (In essence, over the course of the educational part of people's lives, slowly shift the criteria, and help them choose paths that will lead them to success.)

I believe that to learn well, you need to be challenged, but not too much. Ability-based evaluation only does that for students whose abilities happen to line up with the expected standard. It is bad both for gifted students and for struggling students.

4. injidup ◴[] No.41905674[source]
It's funny. You have created yourself a paradox. Replace comp sci with being a teacher. You have made the claim now that teachers can be incompetent but Johnny cannot be. Let's say Johnny wants to become a teacher and puts in lots of effort but just cannot teach. Now he is an incompetent teacher but at what point did he go from being judged on effort to being judged on ability? When he wanted to be a teacher and got a free pass for being a bad teacher? When he went for his first job and got a free pass for failing his exams? When his entire class learned nothing because he was unable to teach even though he put in lots of effort?

Where is the transition? At some point ability is more important than effort.

replies(1): >>41911286 #
5. strogonoff ◴[] No.41911286{3}[source]
The paradox is only in your head. Do not confuse the process of learning a skill and practicing it professionally. The line between the two is beyond clear.
replies(1): >>41911542 #
6. injidup ◴[] No.41911542{4}[source]
The question you refuse to answer is at what point should incompetencey be judged over effort. Little timmy who was always going to be a good teacher has now lost out because you the gave the position for the university place to little Johhny who everyone, despite all his everyone knew he was going to be a terrible teacher.

There is no benefit in always being praised for your efforts if you cannot deliver the goods.