Info

social, health, political imagery through the lens of G J Huba PhD © 2012-2021

Posts tagged Psychometrics

Choose another tag?

Structural equation models, popularized by Joreskog and Sorbom within their LISREL computer program that solved long-standing mathematical estimation issues, have been recognized by many as the most powerful (or one of few most powerful) statistical model(s) available within the social and health sciences. A combination of concept mapping and mind mapping can quite effectively be used as a direct visual (or theoretical) analog of complex mathematical structural equation models. Such maps can be a good way of communicating the results of the structural equation modeling or in developing the theoretical models that will be tested. And yes, you can test the fit (appropriateness) of the models (diagrams) in a statistically rigorous way.

Yup, you heard here first (although path-like or combined mind map/concept models have been around for at least 45 years and I published hundreds of these in the late 70s though the 1990s in peer-reviewed social science and methodology journals and others simultaneously and subsequently have done the same). I am going to blog a LOT on this in the next few months. Mind and concept maps can be directly and formally simultaneously assessed if there are appropriate data available. Of course, such data are hard to come by, but not as hard as many believe.

The best way to draw concurrent concept and mind maps models is #iMindMap (using the flow chart option along with a mind map for a hybrid model). In structural equation modeling, a mind map would represent the estimation of measurement model parameters while the concurrent concept map would represent the structural model.

Mind Maps + Concept Maps can be statistically and concurrently estimated from appropriate data by Structural Equation Models.

Really.

Needs a lot of data, though. Current Macs and PCs are finally fast enough to do the statistical calculations in a notebook computer.

This is Mind Mapping 3.0 at its very best.

Much more on this topic is coming including a lot of demonstrations that mind maps and concept maps can be easily developed as a consequence of very rigorous mathematical model. And tested for goodness-of-fit.

[aaaahhh what the hell … While I am tempted to add a lot of equations, numerical analysis, and map pictures to this post, I will not do so and not mix up the media and message. This is the pilot. The series is coming this summer.]

The following generic sample illustrates what the hybrid models look like.

Constructs are measured by indicators. Indicators are usually imperfect and can be corrected in the statistical modeling. The relationship between constructs and indicators is shown in the mind map.

The constructs themselves may be related in a causal or noncausal way. The concept map shows the relationships that are present as determined by the statistical modeling.

Click on image to increase size.

Topic (correlations among constructs)

Lawley and Maxwell’s book on factor analysis legitimized the psychometric development of factor analysis as a “real” statistical model. Although most now praise them for their breakthrough in deriving maximum likelihood estimators for the model parameters, I think the following sequence of photos shows Lawley and Maxwell’s great insight and most important contribution, received at the time as general heresy by the high priests of factor analysis.

I read this book around 1975. The paragraph I underlined and typed on the cover is one that significantly altered my career: I learned then, and further in the early 1980s, that statistical theories of psychological processes, at their very best, are only weak approximations to reality. At the time lots of psychometricians were giving their professional lives to determine if one blindly empirical factor rotation method was better than another since after all they were different by 3 percent and everyone had just “discovered” Joreskog and Sorbom’s work on structural equation modeling. I never spent more than 50 percent of my research time between 1977 and 1984 on psychometrics and statistics — the rest of the time went into modeling adolescent drug abuse behaviors and their precursors. In 1984 it was time to move on to 90 percent of my work being devoted to real psychological and social issues.

As for statistical-psychology theories, the fact that

  • it’s new
  • it’s hot
  • you can publish it a lot

does not sway me anymore.

Please click on the pictures to zoom.

Image

lawley maxwell 3

NOT

mantra of the failed academic researcher

Please click on the graphics to zoom.

it's program evaluation, not research, dummy

 

3

evaluationmap

researchmap

Big data this, big data that. Wow. At the end we will have better ways to sell underwear, automobiles, and “next day” pills (although in the latter case politics and religion might actually trump Amazon and Google). Blind empiricism. Every time you click a key on the Internet it goes into some big database.

“Little data” — lovingly crafted to test theories and collected and analyzed with great care by highly trained professionals — has built our theories of personality, social interactions, the cosmos, and the behavioral economics of  buying or saving.

Big data drives marketing. Little data drives the future through generalizable theory.

Click on the figure below to zoom.

in praise of little data

This is the first of a series of posts I am making about program-organizational (and individual) evaluation. Much of what I will discuss is not in the mainstream of traditional program evaluation methodology.

My approach is different. It works.

In this first section the point is — obviously — that evaluation is iterative and nonlinear. This led to my first model that EVALUATION IS DETECTIVE WORK several decades ago. [Perhaps that explains my current obsession with all versions of Sherlock Holmes, whether in the original, present London, present New York, or by Iron Man.] At any rate, it seems ELEMENTARY to me that instead of thinking of program evaluation as a linear research experiment with a fixed design (a metaphor that works at best imperfectly), it is more important to treat evaluation as detective work where good rules of evidence must be followed and the evaluator is at fault if all outcomes are not found.

My initial development of the Detective Model in 1992 came from my observation that in much traditional program evaluation the evaluator applies a flawed “research” experimental model and the insensitivity of this approach means that a program looks worse than it is because the evaluation methodology is in error. Who pays for this problem? The program, of course, since the evaluator walks away saying that the “program sucks” and not that the evaluator screwed up. In the Detective Model, applied iteratively and nonlinearly, the evaluator and the program are partners, and it is clear what the responsibilities and level of success each has.

Seems ELEMENTARY to me.

As usual click on the image to zoom.

Evaluation is an Iterative, Non-linear Process

I confess. In 1979 Pete Bentler and I published an article entitled “Simple Minitheories of Love” in the highest prestige journal on personality and social psychology.

Blame it on the exploits of the greatest psychometrician of his generation and a 28 year-old wanna-be psychometrician, both active personality researchers, trying to convince the field that the new statistical modeling methods (Structural Equation Models; LISREL) they were testing would revolutionize the field (I was wrong on that one, too).

Now ask yourself why neither of these guys — nor any of the other main figures in the fields of psychometrics, sociometrics, personality, social psychology, attraction research — ever went on to start a web site to match individuals on the basis of personality and life style questionnaires (I won’t dignify them by calling them tests); such sites became quite lucrative. This was in spite of the fact that at least one (Huba) had the opportunity to do so during the years when he was the Vice President of R&D for a major psychological testing company and later when most of the other competing testing companies hired him as consultant. Or why did the major personality test developer of his generation and the owner of a psychological testing company (the late Doug Jackson) never consider developing such a product?

See a pattern here? Even the folks who made the most $$$ from psychological instruments and had the most influence in the psychological assessment journals and industry did not develop a Love Site.

I concede that a Love Site may be a good place to find people you might not never meet otherwise through your social and work friends and these might be good mates or sex partners. Or they might be psychopaths, perpetuators of sexual or domestic violence, dependent individuals, or alcoholics.

So far as I can tell from the undisclosed algorithms of the dating sites and their unpublished outcomes, I have no way of knowing for sure if the sites have a good chance of producing a good outcome and avoiding a terrible (and life-threatening) one. I suspect that if there were strong scientific evidence that the sites “work” in both cases, there would be a lot of scientific research published that supports this notion. Where is the incontrovertible evidence? Can I can read it or hear it at professional conventions? Claims on TV that a lot of people got married mean little or nothing without information about comparison groups or negative outcomes.

I would have no problem concluding that the Love Sites are effective if there were psychometric and other scientific evidence that the algorithms used are valid. Without such evidence, I worry that they are more voodoo and “smoke and mirrors” than places where you can find a mate and your date will not result in a rape. Of course I cannot prove my position is right, but neither can the Love Sites. My stance is safer for individuals.

There is that old fashioned system of “meet and greet and respect the people you meet” that did produce so many humans that we now have a problem with world-wide population growth. Sometimes older methods work better if you are patient.

Love Sites