Ladies and gentlemen, I wish you all good morning. My talk today is Introduction to Systematic Review and Meta-Analysis with Application to Occupational Medicine. I was asked to review a couple of these points and from last time, and I've made some changes, I want to thank everybody for joining the conference. And I hope to be able to contribute to everybody's use. And I'm going to talk about systematic review and meta-analysis because these types of studies are increasing every day. It is rare indeed now that one can't find a journal without an article talking about this. So the issue is why, and I'm going to talk about and end with a quick and dirty evaluation of a systematic analysis, systematic review and analysis. So here we go. I have no relevant financial interest in this. I have an academic interest only. We're going to describe a systematic review and meta-analysis, present components of such a study using examples from the literature. I'm going to try to explain the components and how to interpret them and apply the structured presentation of a meta-analysis to a published paper. Now I'm going to talk about how to evaluate it, not how to do it. I'm sure I'm the only one in the room or online that gets orgasmic over statistics. So we're going to try to take that out of this, but more on how to interpret them once they are calculated. And in preparation for this, for those in the audience and for those at home, if you'd identify the keystrokes on your computer that we'd use to find a term or a phrase, usually like Control-F for one, and I think it's still Control-F for the max, I'm not sure. And also to identify the keystrokes that will alternate between open windows and your computer, like Alt-Tab, for example, might flip between two open windows. If not, we can still go in seriatim. It'll just be flopping back and forth to highlight certain parts of the paper. So what I'd like you to do online, Ms. Rhonda has posted the PowerPoint presentation. And if you're at home, there's also a paper that you can download called The Effect of Long Working Hours on Overtime on Occupational Health. And I've given the reference down below in case you want to retrieve it. So I want to start with kind of what's called the pyramid of evidence. And as you go up the pyramid, the evidence tends to increase in power and definite generalizability. In vitro research, animal research, but I'm going to walk up in there to ideas, editorials, and opinions. And I cite that now more and more because we find students and other individuals and patients in particular getting their stuff off the social media. And those have no basis. Many of them have no basis in fact, and you have to be careful about it, particularly when they bring in these articles that they've downloaded. Don't consider them to be computer illiterate. They are not. Case reports. Remember, you've done those in the past. And you found an interesting case and reported it. Let me just try to tell you this. The word is, if you've seen one, you've seen one. That's it. I don't want you to start generalizing from an N of one. It's like all Italians eat spaghetti on Wednesday, at least the one I know did. You be very, very careful. The next is a case series, which is I've got more than one. And now we start getting interesting from an epidemiological point of view because now there may be a cluster of certain things that might happen. They're certainly a little more powerful, but they're still single cases. We try to group them relative to the diagnosis and then go from there. Case control studies, recall, is we know what they have, and we're going to go backwards in time to see what got them there. This is great for ethical studies, for example. If you're on a medication which has certain side effects, you don't want to be guilty of inducing those side effects. So it's better to find somebody who has the side effects and then go backwards. It avoids the idea of inducing injury. And then a cohort study is a group, and a cohort was a term used in the Roman times to keep people together. That's the cohort. We form them, and we just march them forward and see what develops. It's great because they're disease-free at the start. The problem is that you may have to follow them for a very long time. And then, of course, near the top was randomized control double-blind studies. Randomized control basically says we get a group of individuals, all of which are disease-free. We shoot them into at least two groups, one doing nothing, one having an active treatment, the placebo versus an active treatment. And we follow them forward in time to see what happens. To give you just a feel of what some of those are, but we randomly assign those people to those groups. So essentially, they're two groups that only differ on what we administer to them. And since they're randomized control, we shouldn't have any inherent biases evident in each one because everybody's gonna carry what they have in and it should be disease-free. But notice that's not the top of the pyramid. And it certainly used to be, but now it is not. Now systematic review and meta-analysis is the one that goes because of the parsing of the articles that have to go into making the argument. And that's why they're, and we now have computer programs that allow us to do this. And this now even steps up the movement for systematic reviews and meta-analysis even more. I'm going to just dash over this for just a bit and allow you a chance to see it, that all clinical questions, we use systematic review. For therapy, we like randomized controlled trials. Diagnosis, we also like randomized, but we can also introduce a cohort. And occupational medicine in particular, cohort studies are very good. Remember, you can get people working in factory online, but we're going to compare building A to building B, or we're going to do section A, section B. This was done a lot in post offices, but people were just going to their only job and then they would leave. And you can compare pretty quickly. Prevention is randomized controlled trial again, but the meta-analysis works up, particularly prospective studies, which is the same as a cohort study. Prognosis and cost, economic evaluation tends to, you're going to do, your dependent variable you're going to be looking at is going to be money, and that's going to be cost per patient, cost per administration, cost per something. Now, first of all, let's think about what a systematic review is. It's sort of a gathering of all the research articles purported to go on the same topic. Now, God, doesn't that just sound appealing? We're going to get all the relevant articles in one place. Neat stuff. And you can remember, I'm sure, when you go back to your medical school days or even after that, times you went back to the library and which book do you want to read? Do you want to read Harrison's? Do you want to read something else? You don't want to read the local journal, latest article. We used to have students still read the latest article, New England J, and discuss it in journal clubs, so on and so forth. Here, it's all together. And they cover different variables, but the focus is on the same outcome. That's the big difference. It's essentially a collection of articles that have a common outcome. Secondly, it's not always associated with systematic reviews, but a meta-analysis is a statistical treatment of the data in these collected articles. The idea is to get an overall result from all the articles you collect. So, based on everything, here's what we can conclude as opposed to a single study. And boy, doesn't that sound delightful. Now, it's an aggregate, in other words, in an overall statistical number of what it is that's going on. It statistically summarizes the results. Now, this is not without criticism as a technique, and it does exclude and it does not exclude or excuse professional judgment. I'll harken back to a talk we had yesterday on AI, and I remember Dr. Wentz asked our speaker when he said that once AI produces results, you have to do a self-analysis of the information. And upon clarification, it was, yeah, the burden fell still on the reader and interpreter of the information. And that's the same stuff with systematic review, and I'll try to show you the paper as an example. Now, so systematic review does this. It attempts to collect all possible studies, present a criteria for selection, assess what's going on, and notice part two, done by more than one person. That's gonna be one of the criterias on what I'm gonna call a quick and dirty. We just don't want some guy doing it on his own. You've got to have inner subjective testability. In other words, two people seeing the same thing. For a meta-analysis, meta here means on the overall, tries to show objective analysis of a combined results, tried to look for any, and to provide a quantitative measure of the outcome for combining the studies. Sir? Doesn't this meta-analysis, doesn't it inherently suffer from publication bias? I'm gonna get to that. Okay. So, first warning. A key limitation to this is that you gotta have to make sure that all of the studies essentially get combined. And this pooled estimate derived from meta-analysis, particularly randomized trials, are certainly low risk of bias, but as we go into other sorts of articles, that bias becomes less protective. Like case control studies. One of the things about case control studies is who's your control group? How about you want to get a control group on exercise? Why don't we use accountants during tax season? You know, these people don't come up for air. You know, they sit in their office and smell bad because you're not gonna get out. And let's compare them to D1 school athletes and find out who exercises more. You know, you've got a hokey control group. And you might say that the active group is still a little different as well. Second warning. There's both a subjective and a statistical component to this type of a study. And where statistics is not gonna save you, there's a subjective component as well. Think for a moment. If you had a collection of articles addressing the same thing, what do you think would cause the results to be different? If you just had five articles and all said the same, but what might cause some of these things to be different that you'd have to worry about? Yeah, one of them is control group. Anything else? Age, how about number of people in the study? You wanna remember that end of one problem? So you've got two things, structural and statistical. Did the, question? Yes, definitions. Is the outcome the same? Is the input the same? Is the intervention the same? They may use the same terms, but I'm not so sure. Is it a prospective, retrospective, or randomly controls trials? And you wanna try to shift that away. You're gonna have to try to understand that each study brings its own problems. Statistically, and that you evaluate professionally, but statistically, did they measure with the same instrument the outcome? You just saw the previous presentation. One uses the cage, the other uses an audit 10. Is that gonna make a difference? Say measure of the input. You say drinks per day. Is it the amount of alcohol in each drink the same? Is it beer versus bourbon? You say that for one of our friends in the audience. How about the number of subjects? Is it a health versus harm issue? And how big a difference do we really want to worry about? So these become statistical problems that has to be adjusted, as well as a structural problem that has to be looked at. And that's how we're going to use statistics that kind of answer that side of the coin. Collection of things that are different, we have to get a word, and that's called heterogeneity. Different things. By strict definition, this is the differences between studies that are not due to chance. It's structural in nature. It's all the things we've mentioned on the previous slides, and this we are going to try to get a handle on. And if it's there, we have to address it. So if we're going to do a systematic review, here's what the steps are. The authors who are going to do it will formulate the question. They'll have an eligibility criteria. I'm going to come back to that in just a moment. They're going to develop a hypothesis about this that might explain any differences between the studies. They'll do a search. They'll screen the titles. They review full text, possibly, assess bias, abstract data. And if they're going to do a meta-analysis, they will generate a summary, look for explanations of the differences between the studies, and give you an overall estimate of what's going on. And that's really the payoff, number three. Give me an overall estimate of what's going on with some confidence. Now, what makes a lot of this possible is the electronic data that's being used. What makes it possible is the electronic searching on your computer. You do it all the time, whether you use whatever search engine you use. You type in distance to Houston Airport. You click in, and you get 15 different ways of getting there, and advantages and disadvantages. You could do the same thing, latest cause and latest treatment for hypertension. You could do that. We can do all of these electronic tools now, instead of going to the library ourselves. That's how I can really jump on a lot of articles in a quick way. And that's what makes one of these types of studies very useful. And in order to do that, we have to have some criteria. And this PRISMA is an acronym for Preferred Reporting Items for Systematic Review and Meta-Analysis. The reason I want to talk about that is, if they don't have a criteria cited, like PRISMA or something else, then I want you to cast sheer doubt on the use of that article. And I'll show you an example of that. And this, essentially, is the checklist. So you can see there are several things that have to go on. I'm not going to read this, but just show you that this is certainly a regimented way that, if you follow the PRISMA criteria, this stuff, you're going to have some assurance that, at least, they tried to check all the boxes. To make myself clear on that, that's just what's. OK. One of the things I want you to look at in a study is this flowchart, which shows how many studies were gathered and how they start to evaluate it. I want you to just take a look at the left-hand line. They started 3,644 studies that met their search criteria. Here comes 3,000 studies. They removed the duplicates, now down to 1,423. They screened it, and they showed you that records excluded due to insufficient data or transformation into odds ratios and so forth, which is great news for me. You don't really care. But the notion is we've now got it down to 83. From there, they went down a little bit more, and they full-text articles by eligibility. They reduced a couple more, and bang, they get to 48. And studies included for the final analysis was 46. Now, think about it. They started with 3,644 articles. After applying their criteria, they end up with 46 articles to review. That's why this is important. Now, the question is, who's doing that? At least two people? Do they have a criteria? Yes, they do. Will they tell you the criteria? Yes, they should. So now you've got the studies. Remember, I said the real payoff here is to get an overall estimate of what's going on. And now we want to try to assess for any biases in these articles. And if bias is found, we have to quantify it and explain it. One of the things we worry about is sample size. Sample size is one of the major problems in assembling these studies. But now I want to stop for one more thing. And we'll come to it later, but let me talk about it now. When we search using your electronic tools, much of what is searched for is published literature. And generally speaking, you're at the mercy of the editors of the journal. What are the odds of getting a non-statistically significant result published? Answer, is it zero? No, but you can see it from there. No, but you can see it from there. All right? So that's called publication bias. And it's biased towards statistical significance and refereed journals. But you've had articles and points. Dr. Filippi has done a couple of that. They're not published, but he uses data. And that data is very good. So the notion then is you have to be able to search sources that may not be published. Many of you belong to societies which produce proceedings. Have any of you been to those? And those are the papers. And we now have it electronically. You can buy the slides. And you can go through all of this stuff if you want. They're not published, but they're available. So did instruments like that also get reviewed? That's not published, but useful. So the question now is, how can we make sure that we've tried to look at all of the sources? Now, sample size is a big one for each study because sample size looks at variance. It helps compute variance. How spread out are your subjects? How tight are your findings? How close do they mimic each other? Things like the effect size, the odds ratios, risk ratios, all of these are affected by sample size. And studies with large samples as compared to small samples reduce variance, yield better estimates, and are more representative of the population in which it is drawn. So sample size is a critical issue. Heterogeneity, remember, that's the difference or multiple variances. Inspect the rules. Make sure that they have addressed the issue of heterogeneity. How did they try to explain why certain studies reduce different results? There's two ways, and I'll show you an example of them, a forest plot or a funnel plot. And I'll show you what those look like. And then there's a statistic for heterogeneity. And that's good news because now we can affect how do the variances between all the studies, these 43 that were looked at, what's going on with that? How does that impact the result? There is a statistics called Higgins I squared. Now, I want everybody to reach into your pocket and look for a quarter. Can everybody do that? Put your hand down there. Now, I'm not asking you to do that. Now, just remember, 25.2. If you found two quarters, how many would you have? 50. You had three quarters, how many you have? 75. That's the rules for Higgins interpretation. So remember your quarter. You remember what it is, 25%, not a problem. 50% is a problem. 75%, we got trouble. And so now you'll always remember what it is. Got it? Now, as we now start moving along, we're going to use this quite a bit. That statistic tells us the percentage of the variability is not due to chance or sampling error. In other words, what's left over once we take chance out of the picture? So a funnel plot merely tries to take a look at all of the data from the studies and see if it's really balanced or shifted. And this is what a funnel plot looks like. You'll notice that in figure A, which is there on the left, funnel plot, they calculated it. And what you kind of see is that see how spread out it is. And you see a couple of circles. Do I have an ability to point on this? Can you all see this? This is outside the funnel. This one's outside the funnel. We've got these things that kind of spread out and so on. And then what they do is they adjust that funnel plot with some statistical analysis. Now, can you see how close these are now to the center? How we tend to eliminate some of the odd ones that are coming closer. We're really reducing ourselves down to a fairly tight cluster as opposed to something which has been a little bit wider. Does that make sense to you? We've tried to essentially hone in and get increased precision in B, which use the same data as A, with some statistical modification. So the funnel plot tries to reduce the spread. We can kind of see how far they go. And we get these things closer and closer and closer to the center line, which is right over here. This is going to be our overall effect, this kind of squash diamond affair. And what we've done is be able to shorten down the distance from that point to this point compared from this one, which is my pre-funnel plot. So a funnel plot is a way of trying to get all of the studies together and then to reduce these heterogeneity to try to reduce that variance to get a tighter estimate of what it is that's going on. So what I've tried to do is talk about that publication bias. When it happens that they publish only, we have to worry about the outcomes. Statistic reviewers use a lot of search engines to locate articles. But this assumes that they are published sort of a circular argument. And we instruct people to look at proceedings, registries, information presented at research meetings like this one. Bottom line is, we have to explain it if we find it. That's the object of the authors. So there's two ways of fixing an asymmetric plot, like I just showed you in the funnel. One's called trim and fill. Trim and fill method aims at estimating potentially missing studies due to publication bias. Fundamental assumption of the trim and fill method is that the studies with the most extreme effect sizes, either on the left or the right or the right side, are suppressed. In other words, if, oops, sorry, I went the wrong way. My fault. What we're trying to do here is these are outside the lines. And so if we knock this one out, we knock this one out, what tends to see what's going on? So if we knock out the extremes, we get a better shot at the middle. And that's where most of our stuff is. Make sense? OK. So this one, and you can see a little bit better. This is a more dramatic. We can see how the spread tends to go, how these clusters tend to develop. And then what we do is we adjust it for bias, and we end up with a different sort of a clustering. They have a tendency to start to come together. And that's, again, a trim and fill on the funnel. What we try to do is reduce the variance, try to even it up on each side. For the next few slides, I'll explain what's going on, but not provide any of the formula, just to allow you to see what it is. Heterogeneity, remember, refers to the differences between study results beyond that which is attributable to chance. It's always present. Question? We have a question? Yes, ma'am. No, we don't necessarily eliminate them, we just see what effect those two studies have on the overall rate. The question was, once you, once you go outside the funnel on the extreme articles, we just throw them away. We don't throw them away, we just see what their impact is on everything, so we can then take that into account by weighting them in the overall estimate. In other words, we won't give them as much weight as we did this stuff on the middle. Now, one statistic I said is I squared, 0.2, 0.5, 0.75. That's how we do it, now you know how to remember it. And now what we're going to do is how do you adjust it? We do that by weighting, weighting studies, give them more weights if they're larger. It's because the formula for variance involves average performance and sample size. Those with larger sample size get larger weights. One method used is an inverse variance, which is, you know, I keep it under my pillow, you don't have to worry about it at all. And just then how to give a better representation of our findings. That's the end result. Now, here's a table. You'll notice you've got some studies on, if we move across the table. You've got eight studies here, experimental events. How many were there? Five out of 60. So we had like 55 didn't show anything in the control group is here. And now what we have is what the weight, and you'll notice that some are weighted heavily, 15, 13, some not so much, 6.9, because of the rarity of the events in the small sample. This had only 13 people in it versus something like this, which has 120 in each group, 240. You could see that that's higher weighted. And here's 150 and 150. They get a much higher rating. So that's how we adjust for these, but just give them the, and we calculate these weights and what they should be relative percentage of the total group. And now what we do is we end up with the risk ratio, which is basically which group did better, the experimental or the control group. And what we have here are these lines here are the confidence intervals. And that's the average right there, the blue dot, 0.42. So if I drop down, that's kind of what we're going to start to see. And this is the line right here of, you can see it's centered on one. And let me just pause my remarks for a minute. I want to do a short review. If I said the experimental group and the control group each got 10 on the questionnaire, was there a difference between the groups? How do you, the answer was no. How do you conclude it? Well, you take one group and subtract it from the other and 10 minus 10 is zero. So there's no difference between it, but you know, there's another way we can compute it as well. What happens if I divided those two numbers? 10 over 10 gives you what? One. So if I've got a data, that's what we call continuous. I can add and subtract the numbers. The units are the same. Then a statement of no difference is how close are we to zero? However, in medicine, we use a lot of ratios. Remember from Sister Rosarita's math class, one number divided by another. And they have to be in the same units. But for example, how about body mass index, right? You've got weight and height, different variables. So I just form a ratio, one number over another and calculate the result. So I can now divide one number over to another and get a ratio. And if I divide that number, then the null hypothesis or no difference means how am I centered on one? If it's a continuous variable, is it centered on zero? That's why I want to draw your attention to that center line, which is effectively one. How close are we to one is my null hypothesis. There's no difference between the groups. So we can certainly do a point estimate, find out how far it is away from the mean, and we can do like your P equals 0.05. And if the probability of that happening is less than 0.05, we say statistically significant and there's a difference. But if it's greater than that, we say no, then it's chance could be making that difference. And therefore, we say the groups are the same. So you have to make sure of the number. And by golly, a confidence interval does that and more. A confidence interval will tell you if that number's in that interval, then it's not significant. It borders all the statistically significant values. So not only does it tell you is it statistically significant, it gives you the range of what you could expect. And now we're going to take a look at that. Let's take a look, if you will, at just this top line. So it's got an endpoint of 0.16. I could drop this line straight down, and it's there 0.16 as it increases going this way. And that is now, can you see the tip of this? Do you see how it just crosses that line? And by God, if I drop that down, that's 1.11. Goes right down to here. And it's over here in numbers, over here diagrammatically. And that center point is 0.42. So all this is is a graphic representation of this number. But, and here's my question. If you look at that line, does it contain the null hypothesis? Yes. And therefore, the statistical results are not significant. Because the null hypothesis is in that interval. So by golly, if it touches that line, you got bonkers. Okay. At least in my neighborhood, we use that term. Anyway, let's go down to the next one. Does it touch the line? Yeah. Remember, we're looking for one. Well, if I go from 0.16 and over 16 cents to $1.11, is a dollar contained in here? Sure. That's a null hypothesis. Not significant. Next one, 0.97 to 4.1, from 97 cents to $4. In the interval? Yep. Not statistically significant. And I agree because it just touches it. Now let's go down to this one. That was study number four. From 4.44 to 44.18. Oh, there it is. Right there. Does that contain the null hypothesis? No. Then that study is statistically significant. Quick and easy. How about this one? This is a very long line. And it goes from whatever it is over here. And as long as I cross that line, not statistically significant. So now I want to combine all these studies because this is the payoff. Remember I said, after all these studies, can I get a single estimate of what the hell's going on? Well, there's that diamond. Oops. It goes from here, 39 cents to $1.76. Is a dollar in that interval? Yep. By God, whatever this study is, then these studies show that there's no statistically significant study and no specific outcome in that outcome variable and your groups. The groups behave essentially the same. Quick and dirty. Any question on that? Okay. Now what they then try to do is to say, well, let's take a look. Total events. Nice. Heterogeneity. Well, here's my I squared. I'm not going to worry about this just yet. Here's my I squared. Remember you're reaching the pocket for a quarter. High rate was what? 0.73 quarters, 75 cents. Is this greater? It sure is. It is 0.88 or 88%. That means we've got too much variance in there and we've got to explain it with me. That's how I use the I square. We've got heterogeneity and by golly, we've got to start looking at it. And so I'm not going to just accept no results just yet. I'm going to go back and see what the authors have to say and explain the difference before we go on. Okay. And then they have the total effect size, which is this. They evaluate that black diamond and you can see the p-value is 0.63. Remember what your p-value generally speaking is 0.05. Is this greater than 0.05? And you'd conclude what? Not statistically significant. And it shows. So everything now is consistent with the numbers or the visual or your interpretation. With me? Easy stuff. Okay. So let me start moving along here. Oh, did I go the right way? Is this going back? What happened? Oh, gee, oh gosh, how did this go? I don't know. See, we should be moving along. Okay, now it's going the right way. No, it's not. Can I just? Let me just. I'm asking you to read these slides. I'm trying to get myself together here. Okay. I've tried to. Those are the confidence intervals just as a quick review. That's the risk ratio. No difference line. Overall summary. Heterogeneity test. Overall test. And that corresponds to that point. Does that make sense now as a quick review for what I've said verbally? Okay, great. So how do you, when you define this bias here, this heterogeneity, what are we going to do? Well, you got to explain it. Most people do then subgroup analysis. That is to say, is there a way I can take my findings and group them smaller? For example, let me look at the men. Let me look at just the women. Because maybe by combining those men-women studies, that introduced too much variance and too much change within the system. That's just called a subgroup analysis. In the paper I'm giving you what it did, and I thank Dr. Wriston for giving me the paper, one of the things it does is it looks at each country, then it looks at what type of problem that they had. And you find the differences between problems. And that helps explain some of the differences as well. Okay, so you start stratifying the studies into groups that make sense for your purpose. Now, after all of this stuff is done, we now have to ask ourselves, okay, after we adjust for it, can we still now understand what's going on? And the answer is yes, you can. There's another statistic called Cohen's d squared, which again, 0.2, 0.5, 0.8. Think of it as your quarters again. You'll still be in the same guidelines. Other tests of overall effect, the authors are going to cite them regardless what the authors use. There'll be a p-value associated with it. And it's going to be your same 0.05 or greater will be our cutoff. So here's my quick and dirty checklist for you. Where the definition is clear, the answer is yes, proceed. Are there at least two reviewers? Yes, proceed. No, discard it. Is there an inclusion and exclusion criteria? Yes, proceed, no, reconsider. Is there a selection criteria mentioned, or whatever? Then yes and proceed, no, then read the selection criteria carefully. Sometimes the editors of the journal want you to combine those. Is there a flowchart to show how many studies were found and used? If yes, proceed, no, be very careful. Is there a summary table? Kind of similar to what I showed you. And if there is, good. If not, not so good. Is there a test for heterogeneity? Yes, proceed, no, throw it out. Do the authors try to assess heterogeneity, usually with I squared? Yes, proceed, no, be very cautious. Because sometimes I tell you in the text without showing you the table. Is there a summary of effect size with a p-value? Yes, proceed, no, don't do it. Investigation of heterogeneity, usually a subgroup. Is there any limitation cited? Yes, proceed, no, discard it. And did the author summarize the findings? Yes, proceed. If not, throw it out. Okay, if they're satisfied, state the statistical results and state your application decision. Well, let's take a look. I'm going to go to this article now. Ms. Rhonda has posted it, you can draw it down. And I've tried to indicate with numbers on the side certain things I want to emphasize. So I'm going to show it to you now, I guess. If you have the article, you can flip to it. And page four, first note was, they use Google Scholar and Medline. And I highlighted in red all the keywords they look for. If you were looking for overtime on occupation, would you use these? Long work hours or overtime. Those were nouns that were used. Occupational health or heart disease. Remember, or means one, the other, or both. And look at all the search terms they use in order to pull up those studies. They've restricted their publications to English. Now, those of you in OCMED, would you be interested in other areas of the world? If you wanted to look at the overtime work, and if you did, did they publish in English? Because if they didn't, there's a bias. That's the only reason for my remark. Okay, and they have 423 papers, and I showed you the flowchart. Here's who they excluded. Studies involving night shift, work schedule, and overtime without providing contract hours or regular working hours, for instance, and that was excluded. So if they didn't meet their criteria, then it, you know, came in, but they said you didn't meet it. Here's the flowchart, which I showed you earlier. That's from the article. Inclusion example, the meta-analysis, only the working hours longer than the reference working hours, and their corresponding operations were included. So at least they had it. I'm not asking you to read it just yet. I'm asking you to use your Ctrl F key to look for inclusion criteria. Use your Ctrl F key to look for exclusion. Look for your Ctrl F key to look for heterogeneity. Ctrl F, remember that's the find function in Microsoft language. Definitions, and let them pop right up so you can see it. I'm not asking you to evaluate the article yet. I'm just asking you to find out if it's there. They used a random effects model for meta-analysis. There they used I squared. We just talked about that. That's good news. The greater the value of I squared, the more considerable the heterogeneity. Value of zero means none. Furthermore, the publication bias of the five effect sizes were tested by the trim and fill analysis. So they even told you how to finish it. That's good stuff. Again, bang, heterogeneity. Ctrl F, heterogeneity. Ctrl F, trim and fill. See if they had it. Here's a table, page seven, note nine. They've given you this. The point's imputed, and here's the I squared for each one. And they take a look at these lower limits and confidence intervals. Take a look at this last study, and NH, whoever the hell they are, coming in, 1.06 to 0.94. And they are doing what? They're looking for the odds ratio here. So is that statistically significant or not? From a dollar, 65 to 94 cents. No, crossed a dollar, didn't it? Not statistically significant. Here, from a 1.1 to 1.4, statistically significant? Yep, because one's outside the interval. Oh, this is toughy. 1.0 to 1.2. If you really want to show results, you know what you do? You carry it to one more decimal point. That would kick it to the other area. I'm just telling you. Okay. 1.07 to 1.3, statistically significant? Yep. 1.04 to 1.2, significant? Yep. Easy stuff, right? Okay. And now, yes, sir, question in the back. This got cut off. That's my point. You can kind of start seeing some dots. So apparently when I got this copied, it kind of chopped it off. You can see a couple of dots there, and that's the side of the letter. They're all the same. The question was, it looked like I shaved the deck here because I only used two decimal points, but that's because of this little white line that runs down through here, knocked off the T on limit and that decimal point. So thank you for noticing it. Please go back to sleep. Now, so now what we're going to try to do is look at the time between I squared and table, and if you don't have it in front of you, I'm just going to skip over these points, but I made it here in writing. It'll be more sense when you look at the article itself, and you should have that article. Funnel plots, give me that. Another one, a moderator analysis was used. P values in the right-hand column. I mean, this isn't going to tell you much without looking at the article, but a meta-analysis in a summary statement. Use 243 records from 46 papers with 814,000 participants in 13 countries. The aggregated odds ratio for the effect of long working hours on occupational health. Now, please, let's all make sure we got the reading the same page. 1.245, is that greater than one? But you can tell nothing until you see the confidence interval or the P value. So now let me look at the confidence interval. 1.195 to 1.298. Is one in that interval? No. That means what? It's statistically significant. Bingo. Yes, ma'am. Okay. The 0.05 is one of the ingredients we use to compute a confidence interval. So a P value of 0.05 means I'm 95% confident that the null hypothesis is in there somewhere. Okay. The question was, what about the 0.05 level? Confidence interval tells you exactly the same Confidence interval tells you exactly the same thing as a P value, except it gives you more. The P value says yes or no. This says yes or no, but gives you the range of possibilities. That's why most journal articles will report both values. Okay. Now, while I'm on it, there's one other thing I'd like to mention on this. Do I have to stand over here? Okay. Here it is. Remember that when an article gets published, you obey the rules of the editor. All of us are familiar with New England Journal of Medicine. If you want to find out the purpose of the paper, what you do is go down to the method section. The sentence or sentences above the method section, the last two, one or two sentences, is where the purpose of the article is stated. And many times it's truncated to give the title of the article to make sure it fits. They'll say the differences between men and women on work environment. That's the title. You go down to the purpose. It says we use extended countries to look at various health indicators relative to working hours. You say, where the hell is that in the title? It's not. So you go down into the article, go control F methods, and then back it up two sentences and you'll find the purpose. And that's where Journal of Occupational Medicine does the same thing. Okay. Quick and dirty. Right or wrong is not an issue here. It's just that's the way the road goes. Question on the floor. Okay. Here now is page 10, which gives you VEC size, the interval, and you see 95% interval, then you'll know that that's a 0.05 level. An alpha level of 0.01 would be a 99% confidence interval. And many drug studies, they want to test at that 0.99, 0.01 because the idea of excluding, pardon me, making a statement is too costly. So they want to be very sure. Some are even advocating 0.001, which opines, I don't do drug trials, but many people find that's too stringent. And if I was doing an exploratory study, I'd test at the 0.10 level. I'd left everything in and then go back and redo it once I get this stuff in, because once we eliminate it, we can't go back to find it. All right, here you go. Males versus females. Odds ratio here. And then once you see their term ratio, we know we're looking at the number one as a nine of no difference. Here's your upper and lower test. They give you a Z value. Here's your P value, 0.000. All three digits are in, I have to note, but the point is, is that less than 0.05? You better believe it. And so is it statistically significant? Yes. Then they tested the model for regression and so on, and by golly, some of that stuff got wiped out a little bit, 0.055. So that's the difference between those two groups. So male and female, at least in this study, once we adjusted for it, didn't seem to have the difference. But look at this one now. Case control studies made a difference. Cross-sectional studies made a difference. And therefore, that's why that starts to jump out. And it's significant as to what type of study was used as to impact on your decision. Similarly, for cutoff points for long working hours, greater than 50 hours per week or greater than 10 hours per day. Less than 50 hours and less than 10 hours, that was a control group, the experimental group and the control group, and we see that that is statistically different. And we take a look at the odds ratio, 1.42 versus 1.07. Clearly, then this group made a difference. And that's where it goes. Question in the back, and I'll repeat it. Question in the back, and I'll repeat it. although we see this thing. Let me try to correct two things. First of all, risk ratio is used for prospective studies. Odds ratios are best used for retrospective studies or case control. Here, what we tried to do is take a look at who was working and then did they come down with, or that would be the great way to look at it, but we can't. What you do is you, otherwise, you'd have to study these people for years. So what you did was look at, did the condition exist, and then go back and see how many hours they worked. That's why the odds ratio, because we look backwards in time, at least this article did, to see if they were working. So we started with the disease and went back to see the exposure. That's why we use an odds ratio. Will the odds ratio approximate the relative risk? And the answer is, yes, it will. It will not be exact. Why? Because there's confounding variables. I go back in time, and a lot of stuff is happening. For a prospective study, they're disease-free at the start. That means I can usually copy or record any of the things which might happen to that person. And I can't do it here, because they're already sick, or else I have to follow them forward for 20 years. Yes, sir? Retrospective studies is odds ratio. Prospective studies, which include usually randomized control studies, cohort studies, we start with them disease-free. That's going to be my relative risk. I start with the group membership or what's going, and I follow them to see if they get the disease. Case control is their case. They have the disease. Now I'm going to go backwards in time to see what they're exposed to. I see the term hazard ratio. Hazard ratio, interpret that as a relative risk. Hazard ratio, the question was, is it a hazard ratio? Hazard ratio is a relative risk. Calculate a little differently. OK, I'm just going to move through some of these, because some of these points I thought were really quite good. Study design, country of origin. It's kind of interesting. You see where the United States lies between some of these high ratios. But I leave you for that to read on the plane, mine of which I found out this morning was delayed. I'm a real. Now, in other words, we've got, they use moderating factors, and they started to find out that white collar, pink collar, and blue collar people, they're different. Yes, ma'am. What's a pink collar? Anybody want to offer, any young men want to offer a definition of pink collar? Question, Rhonda. Yes, ma'am. Generally speaking, female, and also, does it have to do with the sort of job they're doing? Sometimes a lot of it's more of a sense as opposed to production. But blue collar is hard, that kind of stuff. And white collar is physician. No, I'm just kidding. All right, but. Is this a secretarial feature, that kind of stuff? Not exactly. But thank you. Thank you for the question. I just want to make sure that if you all are using it, how you use it, I know how I use it. Yes. And now I don't want to get into the idea of whether or not that's appropriate. I'm saying that's the classification that was used. And for those who are at home, which is saying what is the difference between a white collar, pink collar, and a blue collar occupation? And they have to define that. I'm just going to use it. And that's my bibliography. So let me go back now, having shown you all of that stuff, I want to go back then to this. And the notes that you can see here, where it says page 3, note 1, that you have, I've indicated in red where those are in the article. Remember, the whole idea was to try to use a quick and dirty method to come up with it. I want to finish then with this note, where the definition's clear. When I showed you some of those items, there were at least two reviewers. You saw the inclusion and exclusion criteria. They used Prisma. That was in the article. Was there a flowchart to show what happened to the articles? The answer was yes. Was there a summary table? Yeah. Was there a test for heterogeneity? Remember, that's I squared. Yep. Is there a summary effect size? We saw that Diamond had a p-value with it. But you could tell by, because they graphed it, you could bang, look at it, and tell what's going on as well. Was there investigation of the heterogeneity, like for a subgroup analysis or something like that? Yes, it was. Was the limitation cited? Yes, it was. And did the author summarize the findings? Yes, they did. And the last page I thought was, oh, gosh, I thought I summarized it or pulled it from the article. This meta-analysis synthesizing 243 records given now, first of all, would you believe this article if it was presented to you? Fine. Then let's look at the summary. This meta-analysis synthesizing 243 records from 46 papers with 814,000, you can read that. Bottom line, the aggregated odds ratio for the effect of long working hours on occupational health was 1.245, comprehensible 1.195 to 1.298, which leads us to conclude that there is an effect of long working hours on occupational health. With the subgroup analysis, they then show among the five occupational health conditions, condition of related health showed the strongest association with long working hours. That's what they got out of their subgroup analysis. Health measures in this category were short sleep duration, sleep disturbance, sleep problems, exhaustion, and injuries. Now, come on, folks. Who is that describing? You. And I'm not being facetious. How do you think they tried to lower the residency hours to 40? And how many of you did a residency with 40 hours looked at? Not on my clock. Now, I'm just saying, one of the things that I think is ripe for an analysis like this is medicine. How many of you, when you were doing your medicine, used the word burnout? I'll tell you how many. Very few. How many use it now? Quite a bit. Now, maybe people are sharper today than I was. Certainly, I mean, my social security numbers only got three digits in it. But I'm just trying to tell you that one of the things we have to look at is the effect of hours on health care workers. We saw it demonstrated in the pandemic. We see it demonstrated in our residents. And we see it demonstrated in retirement age of physicians if they feel they can afford to retire. So I really think that that's got to be done. And it's got to be done with the criteria that I mentioned. With that in mind, are there any questions of me? Question in the back? they only were relatively heavyweights in a library like me, and you publish a paper and that's more acceptable than a paper published by a lot of missionaries. Give me a break. Well, the reviewers are those who are collecting the papers, not those that are writing it. In other words, the reviewers are people who have collected. The question on the floor was, where do the two people have to be? Those two people essentially are reviewing the article to see if it should be included in the mass of articles that will end up in the final analysis. They're judges, in other words. They're not journal editors. That's what I meant to say. If that word was confusing, thank you for helping me. Question? Howard? Oh, yes, I'm sorry, Dr. Filgin. It's kind of a comment and a question. You know, as I look at this odds ratio, in my mind, at least, it's fairly low. So would you like to comment? When I look at these meta-analysis and systemic reviews, you know, the strength, the strength of that, am I terming that right? Right, right. Would you comment on that, please? Yes. The question is this. Are we looking for, before I would make that overall effect comment, that overall effect comment says that's permission to snoop. In other words, there's a difference. And now I wanna look at each subgroup to see if it's more effective than others, because some which are less effective is gonna have a smaller contribution than that which is more effective. And I'm gonna look maybe at, not compare, let's say, China to the United States, which may have different rules. Maybe their standard work, for example, are 50, I suppose. So the notion is I would look at the overall odds ratios or the overall estimate as permission to go back and look deeper into the study. And that's then where I would conclude it. And interestingly enough, and then I'll get off. Interestingly enough, we have a new design, not a new design, but one of the designs is you have what we call minimally effective distance. In other words, I will tell how far two groups have to be apart before I'm gonna change my clinical diagnosis, even if it's statistical diagnosis. And that goes on the consumer of the article to say, how big a difference is gonna make a difference? Because that's really your question. Okay, I thank you very much.

systematic reviews

meta-analyses

occupational medicine

methodological rigor

heterogeneity

publication bias

funnel plots

risk ratios

checklist approach

working hours impact