Evaluating Research Reports Chen-Sheng Chen Institute of Rehabilitation Science and Technology National Yang-Ming University September 15, 2003

Analyzing content of an article Title and abstract Introduction Methods Subjects Design Instrumentation Procedures Data analysis Results Discussion and Conclusion

Consideration of merit of an article Reputation of Journal Two experts to review Meet the criteria of journal Elements considered by reviewers Importance of research Originality Appropriateness of research design Adequacy of the method Relevance of discussion Clarity of writing

Science Citation Index 科學情報研究所 Institute for Scientific Information，簡稱ISI 約3500種 理、化、農、林、醫、生命科學、天文、地理、環境、材料、工程技術等自然科學各學科的核心期刊影響因數 影響因數 Impact Factor (IF) 該刊前二年發表的文獻在當年的平均被引用次數

Spine (4/39)

Title and Abstract Title should be informative Information of abstract Next to read abstract Information of abstract Purpose Method Results Major consideration Without studying the details of paper, no way to sure that the study was valid

Introduction Provide the impetus for the study What is the problem? Why is it important? How has the author used the literature to form a logical rationale? What is the theoretical context for the study? What is specific purpose? What are hypotheses or guiding questions?

Methods - subjects Readers need to know who the subjects were, can be applied to clinical situations Who were the subjects? What were inclusion and exclusion criteria? How were subjects selected? How many subjects were studied? Is the sample size adequate?

Inadequate situation Apply the normal to predict the patients Comparison under totally different groups Numbers of subjects is an important consideration when interpreting the results

Methods - Design What is the research design? Is it appropriate for answering the question? How many groups were tested? How often were treatments and measurements applied?

Three reasons to notice First Relate the finding to the ‘real world’ Alert to the possibility that extraneous variable interfered with the results Second Statistical analysis: numbers of groups, numbers of dependent variables, frequency of measurement, etc. Third Clinical trials model or single-case design or, readers should know special rules if can follow up the rules or not

Methods - Instrumentation In instrumentation described insufficient detail? How have the authors documented the reliability and validity of the instruments?

Standardized Questionnaire Reliability Commercial device Model numbers, names, addresses of manufacturers Calibrated regularly Standardized Questionnaire References must be cited Reliability Reproducibility and consistency of measurements Specific populations

Example Test grip strength Reliable to test normal subjects ! Does it reliable to test patients with wrist pain? Does reliability differ if measurement are taken at home or in the clinic? Was the status determined by self-report, interview or observation?

Method - Procedures Are data collection procedures described clearly and insufficient detail to allow replication? What are operational definition for all variables? Position, velocity, acceleration

In vitro test: ligament strength Data collection Who performed the measurement? When or how often measurement were taken? In vitro test: ligament strength Spine specimen Conducted by surgeon Age of specimen Temperature Procedure about removing soft tissue

Data analysis What statistical procedure were used? Are they appropriate to analyze the data? Has the author justified the use of any unique or unusual statistical tests? Has the author addressed each research question in the analysis? What alpha level was used for the level of significance? Inferential statistics, α=0.05

Results Report finding without interpretation Something to notice Do the results address the research question? Are figures and tables presented accurately? Are the results statistically significant? Something to notice Wrong data collection Missing information Subjects -> statistical results

Discussion and Conclusion How does the author interpret results? Did the author clarify if hypotheses were rejected or accepted? How are the finding related to prior reports? What limitations are described? Are there limitations that are not addressed? Does the author discuss how the results apply to practice? Does the author present suggestions for further study? Do the stated conclusions flow logically from the obtained results?

Study for what? What do you do in your study? Why trust your study? Any evidence observed in your results? Inference logically Strictly obey the test procedure To THINK any possibility (台積電: 張忠謀)

管理大師 / 大前研一 顧問費:43萬/1日 核子反應爐工程師 : 9年 麥肯錫東京事務所 圖書室 思考問題解決方案(1year) 核子工程 ??=> 企業管理 損益平衡分析? 公牛乳頭? 圖書室 客戶資料個案微卷 每晚28分鐘的電車 10:48 -11:16 PM 思考問題解決方案(1year) 蕃茄醬廣告 如何擴大某一品牌的蕃茄醬市場? 看板是否能反應銷路? 多元思考 蕃茄醬或蕃茄汁的不同 客戶問題需求 相關思考案例結合 顧問費:43萬/1日

施麗茲啤酒(schlitz beer)行銷顧問 專業知識與製酒過程 清澈密西根湖 自鑽500呎深自流井 5年進行1623次實驗 開發最好的酵母 裝酒瓶: 600度蒸汽殺菌 最「純」的啤酒 告訴消費者過程 體會「純」的定義 排名第一的啤酒 (6 months) 1920年: 十家啤酒廠商 排名第八 啤酒廣告 最純的啤酒 沒人體會什麼是「純」

Strength and Weakness RST 聯強國際總裁:杜書伍 懂80%:常識 專業20:知識 贏20%做老大

Do exercise other than study hard Data adopted from Ch 31. Evaluating Research Report. 鄭誠功所長 : 科技論文寫作 商業週刊 : no. 805 (2003.4.28-2003.5.4) :贏20%做老大 (page 125-131)