ISTE Home
About ISTE
Advocacy
Educator Resources
Membership
Affiliates
All-Inclusives
Awards and Recognition
ISTE 100
Join or Renew
Member Campaigns
Member Central
Member Networking
My Profile
Podcasts
Special Interest Groups
SIG Newsletter
Join a SIG
SIG1to1 (1 to 1 Computing)
SIGAdmin (Administrators)
SIGCT (Computing Teachers)
Join SIGCT
SIGCT Officers
Journal for Computing Teachers (JCT)
JCSE Online - Journal of Computer Science Education
Past Issues
2005-2006
2004-2005
2003-2004
January 2004
November 2003
2002-2003
2001-2002
Submission Guidelines
SIGDE (Digital Equity)
SIGHC (Handheld Computing)
SIGILT (Innovative Learning Technologies)
SIGIVC (Interactive Video Conferencing)
SIGMS (Media Specialists)
SETSIG (Special Education Technology)
SIGTC (Technology Coordinators)
SIGTE (Teacher Educators)
SIGTel (Telelearning)
SIG Council
Volunteer
NECC
NETS
Career Center
News & Events
Professional Development
Publications
Research
Store

Printer Friendly

JCSE Online Survey Results

J. Philip East
University of Northern Iowa

Last spring, the ISTE editorial staff (Davis N. Smith and Kate Conley) and I developed a questionnaire in an effort to assess member satisfaction with JCSE Online and its print compendium JCSE Annual. Other ISTE publications were undergoing similar evaluation. Our goal was to determine how well we were doing our job as we intended and to see how we might improve the journal. There was also the possibility that our members would not consider the journal useful enough to continue.

We developed the questionnaire and, after review by the SIGCS Executive Committee (Pedro Leite, Rich Lamb, Chris Stephenson, and Dennis Jones), placed it online for members to complete via the Web. Only a few members responded. Thinking that we might have missed teachers as they finished up their school year, we asked again this fall (via the SIGCS e-mail list) that members complete our survey. The material below reports those results and my interpretation of their implications for JCSE Online.

1 Response Rate

We received responses from only 11 people. SIGCS has 1,078 members (as of October 10), Thus, the response rate was approximately 1%. Not particularly good. However, only 188 of our members are "standard" members, i.e., they specifically joined SIGCS rather than selecting an all-inclusive membership. Presumably, some of the all-inclusive members are actually interested in SIGCS. I think it reasonable to suggest that we have about 200 to 250 people who are hard-core members of SIGCS. If we had 220 such members, the response rate would be 5%.

Neither the 1% nor 5% response rate is particularly good. They are in line with a similar survey of ISTE’s Learning and Leading publication. I believe they had about 100 respondents from over 10,000 members–a very similar response rate, but a sizeable number by itself. A response rate of 1-5% probably means the results must be assumed to have no validity with respect to representing the overall views of the membership. However, we can use it as data to help guide our actions.

We developed several categories of questions. They overlapped somewhat and, at times, different questions addressed the same issue from slightly different perspectives. We allowed substantial opportunity for ideas beyond the questions and issues we thought of. The issues we did think of addressed general satisfaction with the journal, form and frequency of publication, nature and content of articles published, and general importance of JCSE Online as a part of the activity of SIGCS. The results of these categories is presented below.

 

2 Satisfaction

A major goal of this activity was to determine if our readers were satisfied with the last few years of the publication of the journal. We had come off an unexpected transition in editors and moved to online publishing. The first set of survey questions addressed reader satisfaction with the publication. We asked four questions about reader satisfaction with specifics aspects of journal publication and about their overall satisfaction. We also asked for ideas to enhance their satisfaction. The questions and responses are listed in the table below.

 

Very Satisfied

Somewhat Satisfied

Neutral

Somewhat Unsatisfied

Very Unsatisfied

Please rate your satisfaction with the content of the articles selected

45%

55%

     

Please rate your satisfaction with the frequency of publication

18%

55%

18%

9%

 

Please rate your satisfaction with the regularity of publication

18%

45%

27%

9%

 

Please rate your satisfaction with the form of the publication (i.e., online with print compendium)

73%

 

9%

9%

 

Please rate your overall satisfaction with the journal

27%

45%

9%

   

 

Two respondents had suggestions they thought would increase their satisfaction with JCSE Online & Annual. One person suggested more diverse-teaching oriented material and the other thought increased frequency of publication would help.

When examining questions that have these possibilities, I would expect that if the satisfied customers do not constitute a solid majority, there is a problem. That majority might need to approach 2/3 to 3/4 of the respondents on "satisfaction" items. Using that benchmark, it appears that readers are generally satisfied with the journal but we clearly need to work on frequency and regularity of publication.

 

3 Form and Frequency

The second set of questions asked more specifically about the form (online or print), size (articles per issue), and frequency (issues per academic year) of the journal. The questions had unique responses.

When asked, "Do you prefer online publication, print publication, or a combination of the two?" the responses were:

  • Both: 55%
  • Print only: 18%
  • Online only: 9%

Nine percent did not respond.

Responses to, "How important is a printed version of the journal?" were:

  • Very important: 55%
  • Somewhat important: 18%
  • Neutral: 18%

Nine percent did not respond.

An online publication need not be of a particular size. We wondered what readers might prefer on a continuum of one article per issue, some indefinite quantity called "many", or something in between these two end points of a continuum. The responses were:

  • One at a time: 36%
  • Many at once: 9%
  • Something in between: 55%

We also asked about the regularity of publication, i.e., should we try to maintain a regular schedule or publish material as it became available. The responses in this case were:

  • Regularly: 64%
  • As available: 36%

Finally, we asked how often we should attempt to publish an issue. We try to span the school year with our journal so the maximum number of issues is likely to be nine (September to May). At the other extreme might be one issue per year with , two, three, or four issues seeming reasonable alternatives in between. As one might expect most choices were selected by someone:

  • One: 9%
  • Two: 9%
  • Four: 27%
  • Monthly/nine: 36%

It was suggested that we not try to publish more often than monthly.

Results here suggest that we should maintain the use of online issues with an annual print compendium. Responses about frequency suggest we should try to be more frequent but probably try to publish at least a couple items in each issue. As noted above, regularity is important to the readers.

 

4 Content

The next set of items related to the nature of the content of the journal. We have gone from a newsletter format to a peer-reviewed journal to a relatively formal journal with mostly peer-reviewed articles. At all times, the nature of the journal was largely the choice of the editor (with, with some impact from the submissions received). We were hoping to gain some insight into what content and style our members would prefer in the future.

The first question addressed the "formalness" of the articles we seek to publish. We might wish to publish academic, often research-oriented publications that would often come from writers who teach at the college level. On the other hand we might look for more informal articles that people at all levels might write. The responses here were:

  • Formal: 9%
  • Informal: 9%
  • Both: 82%

Academic journals generally require peer review (refereeing) of the article they publish. We wanted to see how important our readers felt refereed articles were to our journal. First we asked about the quantity of peer-reviewed articles that should be included. The responses were:

  • All: 9%
  • Most: 27%
  • Some: 55%
  • None: 9%

Approaching the same topic from a slightly different angle, we also asked about the importance of including peer-reviewed or refereed articles. The responses were:

  • Very important: 36%
  • Somewhat important: 36%
  • Neutral: 9%
  • Totally unimportant: 18%

We have been republishing articles from other venues relatively often during the last several years. This has been mostly due to not having enough new articles to fill the issues. We wondered if that mattered to the readers and asked about the importance of publishing only original articles. The responses for this question were:

  • Very important: 18%
  • Somewhat important: 18%
  • Neutral: 45%
  • Totally unimportant: 18%

Next we asked about the kinds of articles readers would like to see, suggesting a variety of possibilities. The choices offered and the percentages of respondents who thought each should be included are:

  • CS Ed research: 100%
  • Informal teaching discussions/examples: 91%
  • Reviews of books, applications, programming environments, etc.: 82%
  • Computing tutorials (e.g., how encryption works): 73%
  • Scholarly pedagogical articles: 64%
  • AP CS-related articles: 64%
  • Editorials and opinion pieces: 55%
  • Letters to the editor: 27%

Clearly we should continue publishing refereed academic articles on computer science education. However, we should also be accepting of much other content that relates to teaching computing. Indeed, we probably should seek out such material.

 

5 Importance of JCSE

Finally, we asked about the importance of the journal in relation to the other work currently done by SIGCS. Respondents rating of how important an element of SIGCS’s work the journal is were:

  • Very important: 55%
  • Somewhat important 27%
  • Neutral: 18%

This last item was very gratifying. Over 80% of the readers think the journal is an important part of SIGCS's work. No one indicated it is unimportant.

 

6 Summary

While the results cannot be assumed to represent all readers, I believe we can validly reach several conclusions. We should

  • continue to publish JCSE Online and the JCSE Annual
  • publish more often
  • work much harder to adhere to schedule
  • broaden the content of the journal

In a companion piece in this issue, I discuss my vision for the journal in the near future. That vision includes doing all the above as well as expanding level of coverage from high school to most of K-12.

Customer Service: iste@iste.org   1.800.336.5191   1.541.302.3777 (Int'l)   1.541.302.3778 (fax)
Visit the ISTE Career Center for educational technology jobs, resources, and listings.