Leveraging AI for Designing GIS Workshops: Lessons Learned and Future Directions
by Jack Tieszen
Introduction
A discussion that has occurred since the rise of generative AI programs such as ChatGPT has focused on looking at these systems as learning tools (Resnick 2024). One of many questions that has emerged from this conversation has been: Could someone with little experience gain an expert-level understanding of a subject using these systems? As a GIS librarian, I’ve been curious about this question and its application to my main instruction area of workshops. Utilizing this framework, I posed two additional questions for a generative AI system to test: Was generative AI able to construct a GIS workshop similar to the workshops that I created as a subject librarian? And could one use generative AI to assist students in developing a pathway to learning GIS? I designed these questions to look at the process of learning GIS from a newcomer’s perspective and to see what comparisons there were in the output between generative AI and subject specialists.
The Experiment
The experiment involved the following steps: In ChatGPT version 3.5 and 4, two prompts were input. The first input was: Develop a workshop for teaching students with no experience in GIS about open-source GIS (focusing on QGIS) with a hands-on section in QGIS. This workshop should be around 1 hour in length. The second input, which I provided after getting results from the first, was added to a rewritten first input in the prompt section. This addition was: Give detailed step-by-step instructions for the processes in the hands-on section. I evaluated the output generated by both versions of ChatGPT and compared it to my introductory open-source GIS workshop. Because prompting is a valuable aspect of generative AI, I wrote these two prompts to fit a specific condition. They needed to be detailed enough to provide an output resembling a usable workshop but vague enough to emulate a student who could be considered a GIS novice. The language used in the inputs directly follows these parameters. The experiment was initially performed in May of 2024 and repeated in July of 2024 to provide additional data.
Library Workshop
For comparison, I will first provide an overview of the workshop I teach. This workshop, titled “An Introduction to Open-Source GIS,” is split into two sections: a lecture, which ranges from 15 to 20 minutes, and a hands-on section, which ranges from 40 to 45 minutes. The lecture begins with a basic overview of GIS, discussing key concepts of how these systems work and the types of data they use (vector and raster data). This transitions into a discussion of the types of GIS programs, distinguishing between proprietary and open-source programs that exist and showing examples of each (ArcGIS Pro, QGIS, etc.) to students. The lecture then moves to a discussion of QGIS. From here, I finish the lecture and begin a hands-on section.
The hands-on section is split into two parts: the first is based on learning the basics of the program, and the second is about learning how to produce a usable map with existing data. I start by teaching the QGIS interface, focusing on the location of elements (canvas, toolbars, etc.) and their uses. From there, I discuss using plugins to expand what QGIS can do. I show them how to install plugins such as QuickMapServices, which we use to put an OSM map on the canvas. Using the QuickOSM plugin, we create vector data, and I teach them how to view, select, and edit attributes manually and via queries. From here, we transition to the second section, which involves importing data from an external source. I teach them about the various ways of bringing data into QGIS (browser panel, data source manager), multiple ways of joining the data (vector join, join attributes by value), and discuss the pros and cons of each method. From this, we move to symbology and how to develop a thematic map. I use this section to discuss some of the ethics of map-making and the importance of having a critical eye when visualizing data. Small choices, such as the number of classes for data, can have a massive impact on the look and therefore effectiveness of the overall map, and I emphasize these decisions in the workshop (Monmonier 2005, 217-220). Lastly, we move into creating a map layout, the last element taught before the workshop ends.
ChatGPT Workshops
The original experiment in May yielded this output: Version 3.5 began with an introduction to GIS and a discussion of key concepts such as spatial data, layers, and spatial analysis. It transitioned to an introduction to QGIS, which included an overview of the software and an explanation/navigation tutorial of the interface. Then, it discussed working with spatial data, including a discussion of the various data types, how to import them, and a discussion of attribute tables and symbology. 3.5 then discussed basic spatial analysis, including buffering, overlay analysis, and spatial query. The hands-on section that 3.5 provided consisted of a few sentences stating that “participants will be given a simple spatial analysis task to complete using QGIS” and a few additional suggestions regarding assistance. In response to my second prompt, which requested more detail in the hands-on section, 3.5 incorporated the lecture content into a set of instructions for users to follow.
Version 4 had a similar output to 3.5, though it provided more details. The output included an introduction discussing GIS, what it is and its importance, a discussion of proprietary vs. open-source software, and a section introducing the user to QGIS. This section discussed the interface as well as the basic functions of the open-source program. The hands-on section, which was more in-depth than 3.5’s output, taught loading data, basic map navigation, data visualization, spatial analysis, and creating a map layout.
The experiment redone in July yielded similar results. Structurally, they were the same, with a starting point introducing GIS, moving onto QGIS, and then having a hands-on session afterward. The times designated for the hands-on section were also the same between experiments, with 3.5 suggesting 20 minutes while 4 suggested 30 minutes. While much of the content of each output remained the same between workshops, there were a few differences worth noting, such as 3.5 in July discussing proprietary vs. open-source GIS which 4 did not engage with during this test. 3.5 also included attribute vs. spatial data as a topic of discussion, which none of the other outputs dealt with. Lastly, 3.5 in July provided a hands-on section in its output without needing a second prompt. Regardless of these changes, the overall impact of both versions was similar with regard to constructing a GIS workshop, though version 4 provided more information than version 3.5 during both experiments.
Similarities
From the overviews presented, a reader can notice similarities between these generated outputs and the workshop that I created. Structurally, all the workshops are the same, with a lecture session followed by a hands-on portion. From a content perspective, there is much overlap. All workshops introduced GIS and QGIS; all discussed key concepts such as data types; all addressed navigating the interface of QGIS; and all discussed working with spatial data. There were a few moments where different versions displayed even more similarities, such as when the May ChatGPT 4 and July ChatGPT 3.5 outputs discussed proprietary vs. open-source software, but these additional overlaps were limited. Using these similarities, we can answer the first question of this study: Was generative AI able to construct a GIS workshop similar to the ones I have created? The answer is yes. ChatGPT (and likely other forms of generative AI, provided proper testing) is capable of generating introductory workshops in the field of GIS. However, that does not mean that there are no differences between the subject librarian workshop and what ChatGPT provided.
Minor Differences
Before addressing the significant differences there were a few minor differences between workshops that are worth mentioning due to them providing a good understanding of what ChatGPT’s output prioritizes when it comes to teaching. Some small ones noted upon immediate inspection were the differences in what ChatGPT put in the hands-on section compared to my session. An example of this was that all of the ChatGPT outputs teach buffer analysis, whereas my intro course does not. An additional small difference was that most of the ChatGPT outputs did not address the process of creating map layouts. A more notable difference generated by ChatGPT was in how it designated time for its various sections within the workshop. During both experiments, version 3.5 wanted the hands-on section to be 20 minutes long. Similarly, version 4 suggested a 30-minute hands-on section both times. My version has the hands-on section takes up most of the workshop’s time, with it being 40 to 45 minutes of an hour-long session. This time designation showcases that ChatGPT’s outputs (at least for the prompts inputted) prioritize the lecture style of instruction over hands-on work when no specific teaching style is prompted, despite literature noting that students tend to learn better from active exercises when compared to passive presentations (Ooms et al. 2015, 272). Overall, these differences between ChatGPT’s output and my workshop do not significantly question the effectiveness of ChatGPT as a learning tool because there is no one way to teach GIS. This is true for both content and pedagogical structure, though, as stated, the prioritization of the lecture is worth noting as a possible reason for the generative system being comparatively less effective than a subject expert. However, the major differences between the AI and a subject librarian are more about what ChatGPT omitted rather than what it included.
Ethics in Mapping
A significant difference in the AI output is the absence of discussion involving ethics in data visualization. Within the workshops I design, I try to emphasize how easy it is for maps to be tools of (both intentional and unintentional) misinformation, even if the data presented in them is accurate (Nguyen, Jung, and Gupta 2021, 2). This critical look at mapping not only helps students gain a better awareness of the pitfalls of data visualization and how they see and interpret data, but it also forces them to be more cautious about the choices that they make when they are working with their own data in a GIS program (Webber 2020, 3). Considering that many of the students who take my workshop don’t have a background in data science or GIS, this is something that I think is valuable to teach in an introductory class, as a bad or misleading visualization can lead to incorrect conclusions when interpreting the data (Lisnic et al. 2023, 14 – 15). Not only can this negate good research, but it can result in the spread of misinformation. Even if these students do not use GIS after they have completed the workshop, they will likely continue to see maps or other data visualizations in their lives. Because of this, the value of teaching them about these issues extends far beyond just their use in GIS. ChatGPT did not mention any ethical discussion in any of the variations of this experiment. It mentioned data visualization and spatial statistics but did not address how these elements can be used for manipulative purposes, nor did it warn the user to watch for potential mistakes (Joyeux-Prunel 2013, 4). ChatGPT instructs people on how to make maps but not how to make maps with a critical eye, and this is an issue.
Furthermore, if we expand the discussion to include other research fields, one must ask what else is being omitted from these outputs. ChatGPT is by no means a tool to get a comprehensive understanding of any topic. However, from a student’s perspective, if one does not know these discussions of ethics are even involved in the GIS field, then ChatGPT will be of little help in assisting one with gaining critical insights for effectively using these tools for their research. Whether we are students or instructors, we should all be aware of the ethical issues involved in map-making and should thus be informed of the dangers of ChatGPT’s omission.
Teaching Methods
The other significant difference found in the experiment is concerned less with ethics and more with teaching. There are various ways to teach GIS to beginners, but ChatGPT favors a linear approach. When the generative AI was tasked with giving detailed instructions for the hands-on section, it would provide a step-by-step approach to its outputs. This approach remained the same regardless of which version was used or when it was used. For instance, here is the ChatGPT 4 output for adding vector data was for the May experiment:
– Go to `Layer` > `Add Layer` > `Add Vector Layer`.
– In the dialog box, click `Browse` and navigate to your vector data file (e.g., a shapefile with the extension .shp).
– Select the file and click `Open`, then click `Add`.
This is an accurate way to bring data into the program, but it lacks deeper engagement with QGIS. QGIS, much like other GIS programs, has eccentricities worth teaching to students, such as the various ways in QGIS to accomplish a given task. External data, for instance, can be brought into the program using multiple methods beyond the one that ChatGPT displayed. One can use the layer button in the toolbar, use the data source manager, or drag and drop their data into the canvas. In my workshop, I address multiple forms of various processes because explaining the eccentricities of the system has the potential to ease students into the mindset of a GIS problem solver. Teaching them how to do one thing multiple ways and why one way may work better than another depending on the context provides these students with a deeper knowledge of the system when compared to just teaching them a single process. This knowledge allows them to pivot if something does not work for them. In the same way that a student may not be aware of the ethical considerations of making maps, a student is most likely not aware of diverging pathways in GIS, and telling them that there are alternate ways forward benefits them through deepening their understanding of the system. This is the difference between teaching a process versus teaching a program.
Furthermore, explaining what these students are doing in depth is important, as it displays how particular methods work and the reasons behind using them rather than just explaining the process of getting from point A to point B. For instance, ChatGPT gave instructions for buffer analysis but didn’t explain what buffer analysis was or what situations one would use it for. This approach limits what students learn about the program. When one explains what these students are doing and for what purpose it provides them with enough of a foundational knowledge to begin exploring these programs independently.
Limitations with the Experiment
While this experiment illuminates some gaps between the knowledge of a subject librarian and that of an AI system, some problems need to be addressed. This experiment was only done using ChatGPT, one of the many generative AI programs available to the public. I have not run the experiment with other generative AI systems, so I cannot state if the results would be of similar quality or if the issues addressed would be different with those systems. Another problem with the experiment is that AI technology progresses quickly, so it is hard to determine whether these issues will be solved in future versions or become even more prominent as time passes. This was why I repeated the experiment months after the first test, and while the differences in outputs between the two tests were minimal, it is hard to definitively say that ChatGPT will always provide answers like the ones that were given in this experiment. A different element to address is that I did not specify the modality of the workshop. While I teach my workshops virtually, it is possible that ChatGPT interpreted the prompt to create an in-person workshop, though it did not indicate any such preference in its output. A counterpoint to this is that the modality of the workshop has little impact when looking exclusively at content. Furthermore, the modality of a workshop does not seem to significantly impact those who participate in them, though there are certain elements of a virtual workshop that can be valuable to students such as having easier access to the instructor or the ability to ask questions anonymously (Minn 2024, 7). Lastly, it is important to understand that while ChatGPT’s omissions were a significant point of discussion, it does not mean that ChatGPT is incapable of generating text to address things like ethics or the eccentricities of GIS. It is likely possible to engage ChatGPT with these issues through adequate prompt engineering. However, I did not focus on these possibilities in this experiment due to the GIS novice framework that informed the initial prompts, nor have I tested it outside of the range of this experiment, so no definitive proof can be provided in this text.
Conclusion
This brings us back to the one question that has yet to be answered: Could AI be used to assist students in creating a pathway to learning GIS? Generative AI could assist students trying to learn GIS; however, the omissions discussed would require alternate sources to be utilized in the learning process to become a more effective GIS user. So, similar to the first question, the answer is yes, but unlike the first question, the answer to this requires criticism of the effectiveness of these tools as they exist in their current state. Looking towards the future, whether these problems persist or are solved can have a massive impact on the answers to the two questions. However, regardless of what is to happen, the best course of action is to have a critical eye when using these tools and to make those who wish to use them for their education aware of the issues that have been discussed.
When looking at the benefits of ChatGPT for workshop development, it is clear that, at a surface level, generative AI can provide great content comparable to workshops developed by a GIS librarian. However, when addressing the output developed by the tools, various issues, such as the omission of ethical discussion and a linearity with how it teaches GIS programs, leave a bit to be desired out of these systems. The solution is to think of ChatGPT and other generative AI tools not as comprehensive learning systems but as assistants that can provide the basics while still needing the experts to manage some of the more complicated issues and eccentricities that the world of GIS has to offer.
References:
Joyeux-Prunel, Béatrice. 2013. “Do Maps Lie?” ARTL@S Bulletin 2 (2).
Lisnic, Maxim, Cole Polychronis, Alexander Lex, and Marina Kogan. 2023. “Misleading Beyond Visual Tricks: How People Actually Lie with Charts.” In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, 1–21. Hamburg Germany: ACM. https://doi.org/10.1145/3544548.3580910.
Minn, Michael. 2024. “A Case Study of HyFlex in GIS Classes at the University of Illinois.” Journal of Geoscience Education 0 (0): 1–10. https://doi.org/10.1080/10899995.2024.2362112.
Monmonier, Mark. 2005. “Lying with Maps.” Statistical Science 20 (3). https://doi.org/10.1214/088342305000000241.
Nguyen, Vinh T., Kwanghee Jung, and Vibhuti Gupta. 2021. “Examining Data Visualization
Pitfalls in Scientific Publications.” Visual Computing for Industry, Biomedicine, and Art 4 (1): 27. https://doi.org/10.1186/s42492-021-00092-y.
Ooms, Kristien, Philippe De Maeyer, Bart De Wit, Ruben Maddens, Timothy Nuttens, Nico Van De Weghe, and Soetkin Vervust. 2015. “Design and Use of Web Lectures to Enhance GIS Teaching and Learning Strategies: The Students’ Opinions.” Cartography and Geographic Information Science 42 (3): 271–82. https://doi.org/10.1080/15230406.2015.1011236.
Resnick, Mitchel. 2024. “Generative AI and Creative Learning: Concerns, Opportunities, and Choices.” An MIT Exploration of Generative AI, March. https://doi.org/10.21428/e4baedd9.cf3e35e5.
Webber, Karen L. 2020. “Can You Trust Your Eyes? Learn How to Minimize Misinterpretation of Data Reports and Visualization.” Planning for Higher Education 48 (2): 1–7.
Jack Tieszen
Geospatial Resources Librarian
George Mason University
jtieszen@gmu.edu